Source: VIRGINIA POLYTECHNIC INSTITUTE submitted to
PROFESSIONAL DEVELOPMENT DELIVERY MODEL FOR DODEA LEADERSHIP DOD SCHOOLS - PHASE III
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
TERMINATED
Funding Source
Reporting Frequency
Annual
Accession No.
1011250
Grant No.
2016-39590-25894
Project No.
VA.W-2016-09698
Proposal No.
2016-09698
Multistate No.
(N/A)
Program Code
EDPD
Project Start Date
Sep 1, 2016
Project End Date
Aug 31, 2020
Grant Year
2018
Project Director
Kaufman, E. K.
Recipient Organization
VIRGINIA POLYTECHNIC INSTITUTE
(N/A)
BLACKSBURG,VA 24061
Performing Department
School of Education
Non Technical Summary
The Department of Defense Education Activity (DoDEA) Leadership helps ensure the robust education necessary to support college and career readiness among students at military-connected schools. In support of recent restructuring and the creation of Centers for Instructional Leadership (CIL), the DoDEA requests proposals to design, implement, and evaluate a continuous improvement model to facilitate system alignment. This requires addressing the needs of the DoDEA system above school-level CIL cadre, as well as addressing the needs of districts and schools. To satisfy this need, this project will address three specific goals: Provide the DoDEA with (1) a system-wide, standards-based, collaborative model for continuous school improvement; (2) a Train-theTrainer program and Toolkit for continuous improvement to use with the CIL cadre, districts, and schools; and (3) a comprehensive CIL Evaluation report. In pursuit of these goals, the project team at Virginia Tech (VT) will draw upon models for continuous improvement; princples of adult learning; best practices in educational data-based decision making, and a cycle of preparation, inquiry, action, and evaluation. The Train-the-Trainer learning modules and final Train-the-Trainer program assessment will incorporate locally relevant data related to college and career readiness and STEM leadership to facilitate the strategic goals set forth by the DoDEA and United States Department of Agriculture (USDA). To help ensure relevance and practical application, we have assembled a team of educational leadership experts to serve as an advisory board, and we will engage a highly qualifid team of consultants in the coaching process for the CIL cadre. An independent evaluation team form the American Institute of Higher Education will assess project objectives.
Animal Health Component
0%
Research Effort Categories
Basic
20%
Applied
50%
Developmental
30%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
90160993020100%
Goals / Objectives
Goal 1: DoDEA will have a system-wide, standards-based, collaborative model for continuous school improvement.Objective 1.1: Work with DoDEA to establish an internal project leadership advisory group to advise about project design, implementation, management, and evaluation.Objective 1.2: Identify system-level data sources and organizational structures to inform all aspects of project design, implementation, and evaluation.Objective 1.3: Complete a walkthrough of DoDEA data at system and district levels.Objective 1.4: Deliver a standards-based process and tools for determining the fidelity of implementation data on the CIL program.Goal 2: DoDEA will have a Train-the-Trainer program and Toolkit for continuous improvement to use with CIL cadre, districts, and schools.Objective 2.1: Design standards-based Train-the-Trainer program modules and assessments consistent with Phase 1, as identified in the RFA.Objective 2.2: Design comprehensive assessment plan and accompanying resources for Train-the-Trainer program modules and Toolkit.Objective 2.3: Design, implement, and operationalize a web-based delivery system for training program implementation.Objective 2.4: Deliver hard and digital copies of all instructional and assessment materials needed to support the Train-the-Trainer process and Toolkit.Objective 2.5: Provide coaching and virtual support for further extension of professional learning experiences beyond CILs.Objective 2.6: Integrate College and Career Readiness and STEM related subject matter into Train the Trainer modules.Goal 3: DoDEA will have a comprehensive, three-part evaluation of the continuous development model and associated deliverables.Objective 3.1: Evaluate the continuous improvement model to facilitate system alignment.Objective 3.2: Evaluate the Train-the-Trainer program and corresponding Toolkit.Objective 3.3: Evaluate the effectiveness of CIL cadre as measured through the impact of professional learning on behavior.
Project Methods
The project will progress in 4 stages, beginning with system- and district- wide data walkthroughs and ending with evaluations of the Train-the-Trainer program and Toolkit, as well as the impact of both on CIL behaviors. The project will rely on Senior Key Personnel, the Advisory Board, and DoDEA Leadership to provide feedback and guidance throughout 4 stages of project development. Stage 1 of this project will use a data-based decision-making process and data walk throughs to address the DoDEA's capacity for change throughout the system. Stage 2 of this process, the initial Modules for a Train-the-Trainers program and Toolkit, will focus on the knowledge, skills, and tools that system-level stakeholders (i.e., the CIL cadre) need to acquire about continuous improvement, changing school culture, and leadership. Stage 3 of the process will involve the design, implementation, and evaluation of inquiry-focused Modules that the CIL cadre can use to address change at the level of classroom practice. Finally, Stage 4 of the project will involve the design, implementation, and evaluation of the Modules and Toolkit for the CIL cadre, focusing on the resources for professional learning design that will allow change to become job-embedded and sustainable throughout the DoDEA system. By the end of Stage 4, the DoDEA system will be positioned to move forward with Phase 2 of its continuous improvement process: Establish Strategic Objectives and Initiatives.We will use a backwards design process to design the Train-the-Trainers Program, which will begin by aligning each of the module topics with the appropriate DoDEA and AdvanceEd Standards. Based on these standards, we will create a comprehensive Train-the-Trainers assessment plan that includes the standards-based formative and summative assessments for the program. The sequence of content and activities in each module will follow the prepare, inquire, act, evaluate continuous improvement cycle. The Project Team and coaches will work closely with the Advisory Board and DoDEA Leadership Team to solicit feedback about the design, content, and activities of each module.The Train-the-Trainers program will be implemented using a hybrid program delivery model, comprised of a web-based instructional system and face-to-face activities, to deliver a series of educational modules. Implicit in each Train-the-Trainer module is a continuous cycle of preparing for change, inquiring into instructional practices, acting on the analysis of evidence, and evaluating actions. This inquiry-based approach to each of the module topics will facilitate a process-oriented mindset towards continuous improvement amongst the CIL cadre. To facilitate our capacity to meet project goals, the Project Team will rely on continuous feedback from the DoDEA Leadership Team, Project Team Advisory Board, and locally-relevant data sources that pertain to college and career readiness and STEM integration.Each Train-the-Trainer module will be taught and assessed using a project-based approach. Participants will select data-based problems related to their practices, and apply the preparing, inquiry, action, and evaluation techniques that they learn in the module to the problem. Sample problems that program participants may address in their data based projects include: developing school capacity to address College and Career Ready Standards or integrating STEM activities across the general education curriculum. Experts in instructional leadership and continuous improvement will be available for online coaching with participants throughout the semester as they develop projects. Additionally, the modules will draw from locally-relevant evidence related to college and career readiness and STEM integration to facilitate professional learning of these topics. The modules will rely on whole group, small group, and independent activities. The activities will be based on the best principles of professional development for adult learners. We will operationalize the modules using job-embedded, collaborative approaches to professional learning, like book studies, instructional rounds, lesson studies, and peer coaching groups. Using these models will facilitate the CIL cadre's capacity to later implement these same models with principals and teachers in schools.The Project Team, in conjunction with the Advisory Board and DoDEA Leadership, will also develop a Toolkit for use in the continuous development module. This Toolkit will contain all of the resources necessary to implement the Train-the-Trainer modules. It will also include a resource guide that lists references, websites, research projects, and links to organizations focused on college and career readiness, STEM integration, military families in schools, school improvement, and professional learning design and evaluation. Experts from our Advisory Board and expert coaches will assist in development of this Toolkit.Project objectives will be assessed by an independent evaluation team, the American Institute of Higher Education (AmHighEd), LLC. The AmHighEd team will apply their past experience with design and assessment of total quality systems; auditing quality systems for quality awards and accreditations; assessment and measurement of various processes within academia and industry; evaluation of research articles and research projects; preparation of strategic plans and evaluation of vision, mission and strategic goals of organizations; and advisement of management teams for continuous improvement.The evaluation conducted by AmHighEd will consist of three major components.Component 1 : Systemic AlignmentItem 1. Verify the gaps and opportunities for improvement identified through the Self-Assessment and Data Analysis step with the stakeholders.Item 2. Validate metrics and measures selected as Targets for Improvements with stakeholders and organization's overall strategic plans.Item 3. Evaluate the effectiveness of corrective actions and preventive actions identified during implementation.Component 2 - Evaluation of the Train-the-Trainer program and corresponding Toolkit Item 1. Validate needs diagnosed through interviews or data collection with stakeholders.Item 2. Evaluate alignment of objectives with diagnosed needs.Item 3. Evaluate completeness of learning plans to satisfy the defined objectives. o Item 4. Verify and validate the evaluation methods.Component 3 - Evaluation of the Effectiveness of CIL as Measured through the Impact of Professional Learning tem 1. In addition to evaluating items such as content and trainer proficiency, the evaluator will evaluate behaviors.Item 2. Evaluate the effectiveness of the analysis methods with respect to the pre-determined objectives.Item 3. Evaluate the effectiveness of the improvement plans over time (e.g., examine trends, feedback loops, and validation with stakeholders).

Progress 09/01/16 to 08/31/20

Outputs
Target Audience:The professional learning efforts engaged more than 150 employees of the Department of Defense Education Activity (DoDEA), including all of the personnel for the Centers for Instructional Leadership (CILs), Directors for Student Excellence, District Superintendents, and Community Superintendents, as well as many personnel from DoDEA Headquarters. The program evaluation efforts conducted directly served DoDEA administrators and employees, including all of the CIL personnel; the reports were written with that audience in mind. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Training of professional learning development is directly aligned with project goal two, which involved facilitation of train-the-trainer modules in virtual and face-to-face formats. Events addressed topics such as focused collaboration, instructional walkthroughs, enhanced professional learning, coaching, and data-based decision making. We also prepared professional learning plans and facilitation guides for extending learning to those within DoDEA who were unable to participate in the live events during the course of the project. How have the results been disseminated to communities of interest?As a cooperative agreement, this project involved ongoing, close collaboration with DoDEA, often negotiating fine details to ensure that the project activities are aligned with the immediate and emerging needs of the community of interest. Examples of deliverables include professional learning facilitation guides and related "toolkit" resources in PDF portfolios. Other products include evaluation reports, typically provided as both full reports and summary reports, with the idea that summary reports are appropriate for sharing within DoDEA among communities of interest. Beyond DoDEA, we disseminated results through conference presentations, including annual meetings of the North American Colleges and Teachers of Agriculture (NACTA), the Association for International Agricultural and Extension Education (AIAEE), the Association of Leadership Educators (ALE), the International Leadership Association (ILA), and the International Society for Educational Planning (ISEP). Project results were further disseminated through the following publication outlets: Educational Planning, Journal of Leadership Education, Journal of International Agricultural & Extension Education, and Principal Leadership. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? 1.) System-wide standard based collaborative model for continuous improvement: -Provided frameworks and resources for guiding efforts to strengthen the instructional leadership pipeline, including use of a leadership advisory board. -Refined Innovation Configuration Maps that can be used to guide the CILs with ongoing growth and development through standards-based processes. -Assisted CILs in their data-driven decision making process by providing guidance and feedback on data collection tools. -Provided support and input for CIL and HQ meetings, including walkthrough of DoDEA data. -Facilitated focus group discussions among Principals, Assistant Principals, Superintendents, Community Superintendents, and Instructional Systems Specialists. -Facilitated key informant interviews with DoDEA's senior leaders. 2.) Train-the-trainer program and continuous improvement toolkit: -Delivered instructional resources and feedback reports on the virtual and face-to-face learning modules delivered. -Provided structure and content expertise for professional learning and orientation for new CIL personnel. -Prepared professional learning plans and facilitation guides for extending learning to others within DoDEA, including Instructional Systems Specialists and key personnel at DoDEA Headquarters. -Converted Schoology resources into PDF portfolios that allow for both archiving and ongoing sharing of the developed toolkit for instructional leadership. 3.) Comprehensive CIL evaluation: -Completed professional learning needs assessments of DoDEA CIL cadre, informing the train-the-training program and corresponding toolkit. -Designed and facilitated plans for ongoing data collection and analysis, with particular emphasis on the intersections between the CIL functions and stated priorities. -Delivered comprehensive reports that included analysis of intended outcomes, time-on-task pulse data, and insights from interviews and focus groups with key stakeholders, including both local and senior leaders. 4.) Close collaboration for goal attainment: -Hosted frequent check-in meetings with DoDEA project lead, discussing project related activities, based on a consensus agenda. -Convened "think tank" meetings with leadership development specialists at DoDEA Headquarters and key resource people for instructional leadership expertise. -Provided prompt communication and responsiveness to requests for support. -Coordinated information sharing, including key details for travel and security procedures. -Delivered timely reports, including both draft materials and final documents, in accordance with DoDEA expectations and needs.

Publications

  • Type: Other Status: Published Year Published: 2020 Citation: Kaufman, E.K., & colleagues (2020). Comprehensive Evaluation of DoDEA's Centers for Instructional Leadership: Final Report. Alexandria VA: Department of Defense Education Activity.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2020 Citation: Kaufman, E., Coartney, J., Anderson, J., Mitra, S., & Sen, A. (2020, November 5). Studying Leadership as Praxis: Experience With a Worldwide Education System. Proceedings of the International Leadership Association's Global Conference. Retrieved from http://tinyurl.com/y4h85akg
  • Type: Journal Articles Status: Published Year Published: 2020 Citation: Kaufman, E., Mitra, S., Anderson, J., Coartney, J., & Cash, C. (2020). Leading Collaborative Change in an Educational Organization. Journal of Leadership Education, 19(4), 55-67. doi:10.12806/v19/i4/r5
  • Type: Journal Articles Status: Published Year Published: 2020 Citation: Schiavino-Narvaez, B., Kaufman, E., Schuermann, P. J., & Cannon, M. D. (2020). Build leaders using centers for instructional leadership: Coaching model provides support for all. Principal Leadership, 20(8), 46-49. Retrieved from https://www.nassp.org/2020/04/01/build-leaders-using-centers-for-instructional-leadership-april-2020/
  • Type: Journal Articles Status: Published Year Published: 2020 Citation: Anderson, J. C., Kaufman, E., Coartney, J., Mitra, S., & Cash, C. (2020). Using A Tiered Approach for Implementing Networked Learning Communities: A Case Study on Developing the Capacity of Leaders to Implement Systemwide Innovations within International Training and Development Programs. Journal of International Agricultural & Extension Education, 27(1), 6-14. doi:10.5191/jiaee.2020.27101
  • Type: Journal Articles Status: Published Year Published: 2019 Citation: Kaufman, E., et al. (2019). Planning to Create a Culture of Continuous Improvement with the Department of Defense Education Activity. Educational Planning, 26(4), 5-19. Retrieved from http://isep.info/wp-content/uploads/2019/11/191544-JournalVol26-4_electronic.pdf#page=7
  • Type: Conference Papers and Presentations Status: Published Year Published: 2019 Citation: Kaufman, E., Anderson, J., Coartney, J., Mitra, S., & Ripley, D. (2019). A Transformative Approach to Collaborative Change in an Educational Organization. Proceedings of the International Leadership Association's 21st Annual Global Conference, Ottawa, Canada. Retrieved from http://tinyurl.com/y3s67p8c


Progress 09/01/18 to 08/31/19

Outputs
Target Audience:The program evaluation efforts conducted during this reporting period directly served employees of the Department of Defense Education Activity (DoDEA), including all of the personnel for the Centers for Instructional Leadership. Changes/Problems:As a cooperative agreement, this project has involved ongoing, close collaboration with DoDEA, often negotiating fine details to ensure the project activities are aligned with the immediate and emerging needs of the stakeholders. On a broader scale, we have placed less emphasis on a "train-the-trainer" approach and professional learning events because the Chiefs of Instructional Leadership Development (CILDs) indicated a desire to focus more on the comprehensive evaluation of the CIL. Even still, persistent vacancies and turnover among the CIL personnel has directed some professional learning efforts toward onboarding of new personnel. Furthermore, staffing changes at DoDEA Headquarters have slowed progress with planning and implementation of more comprehensive professional learning curricula. Accordingly, we are requesting an additional 12 months to complete the project objectives. What opportunities for training and professional development has the project provided?Training of professional learning development is directly aligned with project goal two, which involves facilitation of train-the-trainer modules in virtual and face-to-face formats. Events have addressed topics such as focused collaboration, instructional walkthroughs, enhanced professional learning, coaching, and data-based decision making. During year three, the professional learning included an online module to acquaint new CIL personnel with key knowledge and insights on the DoDEA Coaching Model. We also prepared a professional learning plan and facilitation guide for extending learning on the DoDEA Coaching Model to others within DoDEA, including Instructional Systems Specialists and key personnel at DoDEA Headquarters. How have the results been disseminated to communities of interest?As a cooperative agreement, this project has involved ongoing, close collaboration with DoDEA, often negotiating fine details to ensure that the project activities are aligned with the immediate and emerging needs of the community of interest. Examples of deliverables from the past year include a proposed vision for DoDEA leadership Development and a coaching workshop facilitation guide. Other products include regular evaluation reports, typically provided as both full reports and summary reports, with the idea that summary reports are appropriate for sharing within DoDEA among communities of interest. Beyond this, we have disseminated results through conference presentations, including recent annual meetings of the North American Colleges and Teachers of Agriculture (NACTA) and the Association for International Agricultural and Extension Education (AIAEE). What do you plan to do during the next reporting period to accomplish the goals?While we have achieved some progress toward all major goals of the project, the curriculum development aspects (primarily Goal 2) will continue through the fall and winter months, after DoDEA completes the onboarding process for a new leadership development specialist. For the evaluation work (primarily Goal 3), the data collection will continue through the summer and early fall, with analysis and report writing during the winter months. Final communication related to continuous improvement (primarily Goal 1) will occur during spring and summer 2020. Also, we have several outreach activities planned, including presentations at the annual conferences for the International Leadership Association (ILA) and the International Society for Educational Planning (ISEP).

Impacts
What was accomplished under these goals? Summary of Phase 1 accomplishments In order to build a system-wide, standards-based, collaborative model for perpetual improvement, the Virginia Tech team developed a framework for DoDEA's Cycle for Improvement and convened Leadership Advisory Board (LAB) meetings. Weprovided various resources and guides, while alsofacilitatingseveral leadership symposium sessions and small group meetings. In order to develop a system wide common approach, Virginia Tech supported the development of a Learning Walkthrough protocol and coordinated an internal review process. Train-the-trainer modules were delivered through webinars and in-person interactive sessions. A framework for DoDEA's Continuous Improvement Toolkit was implemented via Schoology. The compreshensive CIL evaluation report highlighted professional learningneed assessment and early evidences of changes as a result of professional development activities. Summary of Phase 2 accomplishments As the Centers for Instructional Leadership (CILs) advanced their work, Virginia Tech provided additional templates and protocols, particularly related to evaluation and continuous improvement with DoDEA's efforts for instructional leadership. Professional learning efforts focused primarily on coaching, while also attending to considerations for enhanced professional learning. Virginia Tech facilitated several leadership symposium sessions and small group meetings. Train-the-trainer modules were delivered through webinars and in-person interactive sessions, while archiving resources in an online toolkit available via Schoology. Evidence of program impact was highlighted in the Comprehensive CIL Evaluation Report. Creation of the DoDEA Coaching Model was a significant achievement of project activities in Phase 2. Phase 3 accomplishments Within the major goals of the project, we can identify key accomplishments during the past year. 1.) System-wide standard based collaborative model for continuous improvement: -Provided frameworks and resources for guiding efforts to strengthen the instructional leadership pipeline. -Refined Innovation Configuration Maps that can be used to guide the CILs with ongoing growth and development. -Assisted CILs in their data-driven decision making process by providing guidance and feedback on data collection tools. -Provide support and input for CIL and HQ meetings. -Facilitated focused group discussions among Principals, Assistant Principals, Superintendents, Community Superintendents, and Instructional Systems Specialists. -Facilitated key informant interviews with DoDEA's senior leaders. 2.) Train-the-trainer program and continuous improvement toolkit: -Delivered instructional resources and feedback reports on the virtual and face-to-face learning modules delivered during the prior year. -Provided structure and content expertise for an online module to acquaint new CIL personnel with key knowledge and insights on the DoDEA Coaching Model. -Prepared a professional learning plan and facilitation guide for extending learning on the DoDEA Coaching Model to others within DoDEA, including Instructional Systems Specialists and key personnel at DoDEA Headquarters. 3.) Comprehensive CIL evaluation report: -Delivered progress report that included analysis of intended outcomes from the CIL and HQ meetings, data from time-on-task pulse data collection, and insights from interviews and focus groups with key stakeholders, including both local and senior leaders. -Outlined and launched a plan for data collection and analysis during the final project phase, with particular emphasis on the intersections between the CIL functions and stated priorities. 4.) Close collaboration for goal attainment: -Hosted weekly check-in meetings with DoDEA project lead, discussing project related activities, based on a consensus agenda. -Convened "think tank" meetings with leadership development specialist at DoDEA Headquarters and key resource people for instructional leadership expertise. -Provided prompt communication and responsiveness to requests for support. -Coordinated information sharing, including key details for travel and security procedures.

Publications

  • Type: Conference Papers and Presentations Status: Awaiting Publication Year Published: 2019 Citation: Anderson, J.C., Kaufman, E., Ripley, D., Cash, C., Guy, T., Coartney, J., Mitra, S. (in press). Planning Change: A Case Study on Cooperative Extensions Contribution to Creating a Culture of Continuous Improvement in Educational Programs [Abstract]. Proceedings of the Association for International Agricultural and Extension Educations annual conference, Trinidad.
  • Type: Other Status: Published Year Published: 2019 Citation: Kaufman, E.K. Coartney, J., Mitra, S., & colleagues (2019). Progress Report on Development and Evaluation of DoDEAs Centers for Instructional Leadership- Phase 2. Alexandria VA: Department of Defense Education Activity.
  • Type: Conference Papers and Presentations Status: Awaiting Publication Year Published: 2019 Citation: Anderson, J.C., Kaufman, E., Cash, C., Coartney, J., Mitra, S. (in press). Using the Engelbart Organizational Learning Schema to Frame Professional Learning for College Educators [Abstract]. Proceedings of the North American Colleges and Teachers of Agricultures annual conference, Twin Falls, Idaho.
  • Type: Conference Papers and Presentations Status: Awaiting Publication Year Published: 2019 Citation: Kaufman, E.K., Schuermann, P.J., Cannon, M.D., Coartney, J., Mitra, S., & Anderson II, J.C. (in press). Coaching for Improvement: Developing the DoDEA Coaching Model. Proceedings of the Association of Leadership Educators annual conference, Albuquerque, NM.
  • Type: Conference Papers and Presentations Status: Awaiting Publication Year Published: 2019 Citation: Kaufman, E.K., Mitra, S., Anderson II, J.C., Coartney, J., & Ripley, D. (in press). A transformative approach to collaborative change in an educational organization. Proceedings of the International Leadership Association's annual conference, Ottowa, Canada.
  • Type: Journal Articles Status: Accepted Year Published: 2019 Citation: Kaufman, E.K., Cash, C.S., Coartney, J.S., Ripley, D., Guy, T.M., Glenn, W.J., Mitra, S., & Anderson II, J.C. (in press). Planning to create a culture of continuous improvement with the Department of Defense Education Activity. Educational Planning.


Progress 09/01/17 to 08/31/18

Outputs
Target Audience:The professional learning programs involved employees of the Department of Defense Education Activity (DoDEA), including all of the personnel for the Centers for Instructional Leadership (n=27), as well as many personnel from DoDEA Headquarters (n=11) and some local instructional leaders (n=2). In total, we delivered professional learning programming to 40 different individuals. Changes/Problems:As a cooperative agreement, this project has involved ongoing, close collaboration with DoDEA, often negotiating fine details to ensure the project activities are aligned with the immediate and emerging needs of the stakeholders. On a broader scale, we have placed less emphasis on a "train-the-trainer" approach, because the Chiefs of Instructional Leadership Development (CILDs) indicated a desire to focus more on the learning needs of the CIL personnel, before adopting a turnaround curriculum for them to present to others. Also, DoDEA requested we discontinue the Prepare-Inquire-Act-Evaluate model we originally proposed for the DoDEA Cycle of Improvement. Instead, the request was to use language from Plan-Do-Study-Act (PDSA) model (Langley et al., 2009), since some DoDEA personnel were already comfortable with that language. There was also interest in Kotter's change process, so we incorporated Kotter's (2014) model for accelerating change. We have already incorporated the accelerate and PDSA models into our work, including our evaluation framework, which has been aligned with the CIL theory of action. The ultimate question is: "Do the CIL efforts for improved instructional leadership result in improved student achievement?" Another program change relates to staffing of the CILs and the Virginia Tech project team. While the CILs have made progress with filling vacancies, departures of some CIL personnel have kept them at staffing levels of about 50%, which has implications for onboarding new personnel and building upon aspects of the program (e.g., professional learning). Among the Virginia Tech project team, the original project director (Patrizio) has been unable to continue with the project, so Kaufman has worked to find others who can fill the void; including an increase in responsibilities for Carol Cash, key project personnel, and the hiring of consultants outside of Virginia Tech. What opportunities for training and professional development has the project provided?Training of professional learning development is directly aligned with the project goal two which involves facilitation of train-the-trainer module in virtual and in person formats. Events address topics such as focused collaboration, instructional walkthroughs, enhanced professional learning, coaching, and data-based decision making. Highlights of professional learning events during year two of the project included: Five-day professional learning session on coaching with the Center for Instructional Leadership (CIL) personnel in Fort Bragg, North Carolina, with 30 DoDEA participants. Two, 90-minute virtual check-ins with each CIL for professional learning for coaching, with 21 DoDEA participants in the first check-in and 24 DoDEA participants in the second check-in. Three regional visits, one for each Center for Instructional Leadership, as a third follow-up to the session on professional learning on coaching, with 24 total DoDEA participants. Five-day session on Enhanced Professional Learning at Point Loma Naval Base in San Diego CA, with 32 DoDEA participants. How have the results been disseminated to communities of interest?As a cooperative agreement, this project has involved ongoing, close collaboration with DoDEA, often negotiating fine details to ensure that the project activities are aligned with the immediate and emerging need of the community of interest. Products include curricula and module resources to support professional learning topics, including innovation configuration maps and job aids that contribute to an instructional leadership toolkit. Examples of such deliverables are as follows: Job Aid: Overview of Key Elements of Innovation Configuration Map for the DoDEA Coaching Model (Draft Date: February 12, 2018) Innovation Configuration Map for Enhanced Professional Learning for DoDEA's Centers for Instructional Leadership (Draft Date: May 27, 2018) Innovation Configuration Map for Functions of DoDEA's Centers for Instructional Leadership (Draft Date: June 16, 2018) Innovation Configuration Map for DoDEA Coaching Model (Draft Date: June 20, 2018) Professional Learning Initiative Root Cause Worksheet (Draft Date: June 18, 2018) Professional Learning Implementation Drivers Challenge Themes (Draft Date: June 19, 2018) Enhanced Professional Learning by Design Template (Draft Date: June 18, 2018) Shared Agreements and Commitments on Enhanced Professional Learning (Draft Date: July 17, 2018) Other products include regular evaluation reports appropriate for sharing with stakeholders. When providing reports to the DoDEA Project Lead, we provided full reports to the DoDEA personals. Examples of reports delivered during year two included: Progress Report on Development of DoDEA's Centers for Instructional Leadership, Phase 1, September 2016 through August 2017 (Report Date: January 17, 2018) Report of Post-Session Feedback on Coaching [February 2018 session at Fort Bragg] (Report Date: March 6, 2018) Report of Post-Session Feedback on Coaching Webinar 1 (Report Date: April 5, 2018) Report of Post-Session Feedback on April Coaching Webinar (Report Date: April 26, 2018) CIL Time-on-Task Pulse Data Report (Report Date: May 29, 2018) Professional Learning Implementation Drivers Self-Assessment & Status Report (Draft Date: June 18, 2018) Enhanced Professional Learning Reports from Group Projects Related to Implementation of the Learning Walkthrough (Report Date: June 25, 2018) Report of Post-Session Feedback on Enhanced Professional Learning Meeting (Report Date: July 17, 2018) What do you plan to do during the next reporting period to accomplish the goals?The RFA for renewal/continuation funding indicated the following: "The primary objective for the second option year of this project is to continue the development and support of DoDEA Leadership (CIL cadre) addressing the content from the previous years' implementations with a focus on coaching. A second key objective is the gathering of program implementation and impact data using multiple tools to develop a comprehensive program evaluation process and report. A third objective is the consultation and planning for a Leadership Academy concept. Finally, there is a need to continue the development of resources, to support the design, delivery, and implementation of professional learning and archiving these resources in Schoology." In support of this work, we propose continuation of regular meetings and dialogues with key personnel. In particular, we are committed to the following: Facilitating weekly check-in meetings with the DoDEA project lead. Contributing to bi-monthly meetings between the CILDs and DoDEA Headquarters. Refining and archiving documents and multi-media associated with the instructional leadership toolkit for facilitated and self-guided learning within DoDEA. Facilitating bi-weekly meetings with DoDEA personnel who will be responsible for implementing a leadership academy that serves as an effective pipeline for DoDEA's improved instructional leadership. Organizing face-to-face consultation and planning meetings to design a robust leadership academy for DoDEA, based on research and best practices. Participating in monthly collaboration meetings with DoDEA's educational researchers, allowing for close attention to data analysis in ways that uncover the impact of the CILs. Gathering CIL program implementation and impact data from various stakeholders using multiple tools and measures. Completing a comprehensive evaluation of the CILs', highlighting progress with the CIL functions and Kirkpatrick Level 3 behavior change among instructional leaders.

Impacts
What was accomplished under these goals? Within the main goals of the project, we can identify key accomplishments during year two: 1.) System-wide, standards-based, collaborative model for continuous improvement. 1.a) Incorporated Kotter's (2014) accelerate model into the framing and assessment of DoDEA's restructuring for student achievement. 1.b) Provided template and protocol for presenting CIL Quarterly Reports, which helps the CILs to focus on identified priorities and frame their work in terms of the PDSA model for continuous improvement. 1.c) Contributed to discussions in bi-weekly meetings between the CILDs and DoDEA Headquarters. 2.) Train-the-trainer program and continuous improvement toolkit to use with CIL cadre, districts, and schools. 2.a) Developed a DoDEA coaching model and related Innovation Configuration Map (ICM) to guide CIL personnel in their work with instructional leaders. 2.b) Facilitated a week-long professional learning session on coaching with 30 DoDEA personnel, resulting in a 32% gain in proficiency (based on post-then self-assessment on the learning outcomes). 2.c) Facilitated two 90-minute webinars (i.e., virtual check-ins) to follow-up on CIL members application of the coaching model. 2.d) Conducted regional visits with each CIL (Europe, Pacific, and Americas) to further advance their work in coaching instructional leaders. 2.e) Developed protocols on enhanced professional learning, including a related ICM, to help CIL personnel advance their comprehensive approach to professional learning. 2.f) Facilitated a week-long meeting on enhanced professional learning with 32 DoDEA personnel, resulting in a 18% gain in proficiency (based on post-then self-assessment on the learning outcomes). 2.g) Developed the following professional learning resources within the Schoology site: 2.g.i) Self-guided professional learning modules based on the webinars and face-to-face experiences provided to date. 2.g.ii) Instructional Leadership toolkit that provides job aids, exercises, and tools the CIL personnel can use to provide professional learning at the district and school levels. 3.) Comprehensive evaluation of the CILs to inform decision-making. 3.a) Delivered a 45-page "Progress Report on Development of DoDEA's Centers for Instructional Leadership: Phase 1," including evidence of accelerators for change and recommendations for moving forward. 3.b) Provided evaluation question measures grid to frame assessment of the CILs in terms of CIL functions and AdvancED's system quality factors. 3.c) Developed an ICM for the CIL functions and facilitated focus group sessions with the CIL personnel regarding their progress with each function. 3.d) Assisted CILs in their data-driven decision making process by facilitating a time-on-task pulse data collection process that promotes important discussions about how the CILs are fulfilling their mission through ongoing strategic investments and alignment. 3.e) Conducted key informant interviews with individuals who are keenly aware of the design and process guiding the CILs in their efforts to improve instructional leadership. 3.f) Facilitated focused group discussions among stakeholders of the CILs. 4.) Collaborate closely with DoDEA to achieve all project goals. 4.a) Engaged in post-award kick-off meetings to clarify project deliverables, and have worked to adapt as new considerations emerged. 4.b) Hosted weekly check-in meetings with DoDEA Project Lead, discussing project related activities, based on a consensus agenda. 4.c) Shared project-related resources through DoDEA's existing Learning Management System (i.e. Schoology), ensuring the organization and final form of these resources are meeting the needs of DoDEA. 4.d) Fulfilled DoDEA Project Lead requests related to delivery dates for participant programs, travel and security procedures, points of contact at project program delivery sites, and coordination of travel arrangements. 4.e) Provided prompt information related to program delivery sessions, travel and security procedures, points of contact at project program delivery sites, and coordination of travel arrangements. 4.f) Negotiated considerations for target audience and participants in both professional learning and evaluation activities.

Publications

  • Type: Other Status: Other Year Published: 2018 Citation: Kaufman, E. K., Coartney, J.*, Mitra, S.*, & colleagues (2018). Progress report on development of DoDEAs Centers for Instructional Leadership  Phase 1. Alexandria VA: Department of Defense Educational Activity.
  • Type: Websites Status: Published Year Published: 2017 Citation: Kaufman, E. K., Coartney, J.*, Mitra, S.*, & colleagues (2017). Virginia Tech partners with Department of Defense schools to bolster educational leadership across the globe [news release]. Available at https://vtnews.vt.edu/articles/2017/10/cals-defenseschools.html


Progress 09/01/16 to 08/31/17

Outputs
Target Audience:The professional learning programs involved employees of the Department of Defense Education Activity (DoDEA), including all of the personnel for the Centers for Instructional Leadership (n=21), Directors for Student Excellence (n=3), District Superintendents (n=8), and Community Superintendents (n=18), as well as many personnel from DoDEA Headquarters (n=51). In total, we delivered programming to 101 different individuals. Changes/Problems:As a cooperative agreement, this project has involved ongoing, close collaboration with DoDEA, often negotiating fine details to ensure the project activities are aligned with the immediate and emerging needs of the stakeholders. On a broader scale, an example of a negotiated investment is the creation and refinement of a Learning Walkthrough Tool and Guide that will be applied DoDEA-wide beginning in January 2018. The idea for a common Walkthrough emerged through Virginia Tech's facilitation of strategic planning during the leadership symposium in September 2016, and the ongoing work extended through professional learning activities facilitated during the leadership symposium in June 2016. Another example of changes to anticipated program delivery is the inclusion of DoDEA Headquarters (HQ) personnel in many of the professional learning modules. While HQ was not originally identified as a target audience, we added a three-hour face-to-face workshop delivered exclusively for HQ personnel, and we facilitated two 90-minute webinars for DoDEA's Assessment Task Group, which was composed primarily of HQ personnel. These changes have helped to promote a common language and a broader approach to continuous school improvement than was originally articulated. Beyond adjustments to plans for delivery of professional learning, there were also significant changes to plans for evaluation. As we learned more about the evaluation needs associated with the project and the relationship with the expertise of the external evaluation group (AmHighEd), it became apparent the original team and approach was going to be insufficient. We terminated our contract with AmHighEd and requested a no-cost extension on the project to allow additional time for the comprehensive CIL evaluation report. In some ways, the extended time may align with best practice, because DoDEA is still working to fully staff the CILs, which in turn dictates the need for more time to generate and share artifacts for consideration in the evaluation. What opportunities for training and professional development has the project provided? Training and professional development is directly aligned with the project goal two, which involves facilitation of train-the-trainer modules in virtual and in-person delivery formats. Highlights of professional learning offerings include: Three sessions on data-based decision-making and strategic planning for 2016 DoDEA Leadership Symposium at the Joint Expeditionary Base Little Creek-Fort Story, Virginia, with 38 participants. Two sessions on strategic planning with DoDEA CIL Chiefs (and a few headquarters personnel) in December 2016. Three, three-hour webinars for Module 1, "The Role of Professional Learning Communities in Continuous Improvement," with 64 participants. Three, three-hour webinars and one in-person workshop for Module 2, "Developing Capacity with Professional Learning Communities," with 69 participants. Three, three-hour webinars for Module 3, "Implementing Change with Professional Learning Communities," with 68 participants. Three, five-day, on-site Professional Learning Institutes on "Instructional Leadership Skills for Continuous Improvement," with 48 participants. Two 90-minute assessment webinars with DoDEA Assessment Task Group in May 2017. Multiple sessions on continuous improvement skills for instructional leadership for the 2017 DoDEA Leadership Symposium at Fort Benning, Georgia, with 67 participants. How have the results been disseminated to communities of interest?As a cooperative agreement, this project has involved ongoing, close collaboration with DoDEA, often negotiating fine details to ensure the project activities are aligned with the immediate and emerging needs of the communities of interest. One change to anticipated program delivery was the inclusion of DoDEA Headquarters (HQ) personnel in many of the professional learning modules. While HQ was not originally identified as a target audience, we added a three-hour face-to-face workshop delivered exclusively for HQ personnel, and we facilitated two 90-minute webinars for DoDEA's Assessment Task Group, composed primarily of HQ personnel. These changes have helped to promote a common language and a broader approach to continuous school improvement for all of DoDEA's communities of interest. When providing reports to the DoDEA Project Lead, we have typically provided both full reports and summary reports, with the idea that the summary reports are appropriate for sharing within DoDEA among communities of interests. Examples of these reports include: Summary Report of Post-Session Assessments for Modules 1-3 [Report Date: May 18, 2017] Summary Report of Post-Session Assessments for Spring 2017 Professional Learning Institute [Report Date: May 18, 2017] Summary Report of DoDEA CIL Professional Development Needs Assessment [Report Date: June 9, 2017] Insights on Progress from DoDEA Leadership Advisory Board (LAB) for the Centers for Instructional Leadership (CILs) [Report Date: July 18, 2017] Summary Report on Insights from Focus Groups with DoDEA's Centers for Instructional Leadership [Report Date: August 20, 2017] What do you plan to do during the next reporting period to accomplish the goals?During the next reporting period, we will do the following to accomplish the phase one project goals: 1. Continue established communication processes with DoDEA. 2. Confirm final deliverables and process the necessary project re-budget. 3. Finalize structure for Instructional Leadership Toolkit and ensure all requested tools from phase one of the project are reflected in the Toolkit. 4. Finalize project evaluation components of phase one.

Impacts
What was accomplished under these goals? Within the major goals of the project, we can identify key accomplishments to date: System-wide, standards-based, collaborative model for continuous school improvement. Conveyed a structure and support for guiding improvement efforts, including: Presenting framework to serve as DoDEA's Cycle for Improvement (Figure 2). Facilitating leadership symposium sessions on "Continuous Improvement, Theories of Action, and Strategic Planning"; "Data Walkthroughs and Root-Cause Analysis;" and "Continuous Improvement Skills for Instructional Leadership." Facilitating small group meetings on "Planning for Success"; "An Appreciative Approach to Improvement"; "Assessment Inventory: Optimizing the Portfolio of Purposes"; and "Assessment-Related Ideas that Everyone Almost Certainly Already Knows but Might Benefit from Discussing." Providing templates and protocols for theories of action, root cause analysis, strategic planning, action planning, and marketing through DoDEA-wide CIL newsletter. Convening Leadership Advisory Board (LAB) meetings to facilitate broad involvement in planning processes. Provided comprehensive support for development of a Learning Walkthrough Tool and Guide, including: Sharing tools and resources to guide a common approach. Facilitating internal reviews of preliminary drafts of the Tool and Guide. Facilitating leadership symposium sessions on "Learning About the Learning Walkthrough: Components and Implementation." Delivering a complete 44-page PDF guide, including job aids and sample reports. Train-the-Trainer program and Continuous Improvement Toolkit to use with the CIL cadre districts, and schools. Provided structure and guidance for practical approaches for professional learning by shaping professional learning content with the following: Train-the-trainer curriculum maps, instructional guide template; professional learning needs assessment; protocol for feedback loops; and professional learning evaluation instrument. Facilitated train-the-trainer modules in virtual and in-person delivery formats, including: Three, three-hour webinars for Module 1, "The Role of Professional Learning Communities in Continuous Improvement," with 64 participants. Three, three-hour webinars and one in-person workshop for Module 2, "Developing Capacity with Professional Learning Communities," with 69 participants. Three, three-hour webinars for Module 3, "Implementing Change with Professional Learning Communities," with 68 participants. Three, five-day, on-site Professional Learning Institutes on "Instructional Leadership Skills for Continuous Improvement," with 48 participants. Provided a framework and initial content for DoDEA's Continuous Improvement Toolkit to be presented and maintained via Schoology. Comprehensive CIL evaluation report. While the comprehensive CIL evaluation report is forthcoming, we have already prepared components, and the data has been insightful for guiding the ongoing work: Key considerations emerged from discussions with DoDEA CIL LAB: Findings highlight early evidence of changes occurring as a result of professional development activities and investment of time and energy of CIL personnel. Investment to build awareness and trust with instructional leaders has both immediate and long-term benefits for establishing a solid foundation for learning and leading continuous improvement. Formal and informal networks are starting to emerge as a result of professional development, expanding communication channels and building new ones. Systematic data-informed processes are emerging and contributing to decisions about professional learning. The alignment of core values and desired results are having an impact on the daily work of instructional leaders as DoDEA builds the capacity for the "One DoDEA" vision. The module essential questions yielding the most change in knowledge were: How can CIL leadership turn around module 1 and support the embedded work associated with collaborative practices in a way that shifts the culture from one of teaching, to one that is about student learning? [0.9 change related to module 1] Why are protocols in the PLC process important, and when is it appropriate to use them? [0.8 change related to module 2] What opportunities do learning walkthroughs present to facilitate collaboration between schools, districts, and CILs? [0.7 change related to module 4e] How do the 3 big ideas of PLC and the 4 critical questions of a team drive the work of collaboration? [0.7 change related to module 1] How can the CIL help school leaders to leverage knowledge about the change process? [0.7 change related to module 4a] The Professional Development Needs Assessment revealed CIL priorities: Respondents appreciate focused and timely professional learning experiences. The competencies with highest priority for professional learning are in the areas of "Professional Development" and "Facilitative Leadership Skills." The most recent data reveals the highest priority competency is "Implement Professional Learning Experiences," followed by "Cognitive Coaching." Collaborate closely with DoDEA to achieve all project goals. We engaged in post-award kick-off meetings to clarify project deliverables, and we have worked to adapt as new considerations emerged. We convened weekly check-in meetings with DoDEA Project Lead, discussing project related activities, based on a consensus agenda. We have used the existing DoDEA LMS structure (i.e. Schoology) as the platform for sharing project-related resources and are in the process of ensuring the organization and final form of these resources are meeting the needs of DoDEA. We worked with DoDEA to convene three meetings of a DoDEA Leadership Advisory Board, which provided input related to the project deliverables. We fulfilled DoDEA Project Lead requests related to delivery dates for participant programs, travel and security procedures, points of contact at project program delivery sites, and coordination of travel arrangements.

Publications