Source: VIRGINIA POLYTECHNIC INSTITUTE submitted to
PROFESSIONAL DEVELOPMENT DELIVERY MODEL FOR DEPARTMENT OF DEFENSE EDUCATION ACTIVITY LEADERSHIP DEPARTMENT OF DEFENSE SCHOOLS
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
EXTENDED
Funding Source
Reporting Frequency
Annual
Accession No.
1011250
Grant No.
2016-39590-25894
Project No.
VA.W-2016-09698
Proposal No.
2016-09698
Multistate No.
(N/A)
Program Code
EDPD
Project Start Date
Sep 1, 2016
Project End Date
Dec 31, 2017
Grant Year
2016
Project Director
Kaufman, E. K.
Recipient Organization
VIRGINIA POLYTECHNIC INSTITUTE
(N/A)
BLACKSBURG,VA 24061
Performing Department
School of Education
Non Technical Summary
The Department of Defense Education Activity (DoDEA) Leadership helps ensure the robust education necessary to support college and career readiness among students at military-connected schools. In support of recent restructuring and the creation of Centers for Instructional Leadership (CIL), the DoDEA requests proposals to design, implement, and evaluate a continuous improvement model to facilitate system alignment. This requires addressing the needs of the DoDEA system above school-level CIL cadre, as well as addressing the needs of districts and schools. To satisfy this need, this project will address three specific goals: Provide the DoDEA with (1) a system-wide, standards-based, collaborative model for continuous school improvement; (2) a Train-theTrainer program and Toolkit for continuous improvement to use with the CIL cadre, districts, and schools; and (3) a comprehensive CIL Evaluation report. In pursuit of these goals, the project team at Virginia Tech (VT) will draw upon models for continuous improvement; princples of adult learning; best practices in educational data-based decision making, and a cycle of preparation, inquiry, action, and evaluation. The Train-the-Trainer learning modules and final Train-the-Trainer program assessment will incorporate locally relevant data related to college and career readiness and STEM leadership to facilitate the strategic goals set forth by the DoDEA and United States Department of Agriculture (USDA). To help ensure relevance and practical application, we have assembled a team of educational leadership experts to serve as an advisory board, and we will engage a highly qualifid team of consultants in the coaching process for the CIL cadre. An independent evaluation team form the American Institute of Higher Education will assess project objectives.
Animal Health Component
40%
Research Effort Categories
Basic
30%
Applied
40%
Developmental
30%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
90160993020100%
Goals / Objectives
Goal 1: DoDEA will have a system-wide, standards-based, collaborative model for continuous school improvement.Objective 1.1: Work with DoDEA to establish an internal project leadership advisory group to advise about project design, implementation, management, and evaluation.Objective 1.2: Identify system-level data sources and organizational structures to inform all aspects of project design, implementation, and evaluation.Objective 1.3: Complete a walkthrough of DoDEA data at system and district levels.Objective 1.4: Deliver a standards-based process and tools for determining the fidelity of implementation data on the CIL program.Goal 2: DoDEA will have a Train-the-Trainer program and Toolkit for continuous improvement to use with CIL cadre, districts, and schools.Objective 2.1: Design standards-based Train-the-Trainer program modules and assessments consistent with Phase 1, as identified in the RFA.Objective 2.2: Design comprehensive assessment plan and accompanying resources for Train-the-Trainer program modules and Toolkit.Objective 2.3: Design, implement, and operationalize a web-based delivery system for training program implementation.Objective 2.4: Deliver hard and digital copies of all instructional and assessment materials needed to support the Train-the-Trainer process and Toolkit.Objective 2.5: Provide coaching and virtual support for further extension of professional learning experiences beyond CILs.Objective 2.6: Integrate College and Career Readiness and STEM related subject matter into Train the Trainer modules.Goal 3: DoDEA will have a comprehensive, three-part evaluation of the continuous development model and associated deliverables.Objective 3.1: Evaluate the continuous improvement model to facilitate system alignment.Objective 3.2: Evaluate the Train-the-Trainer program and corresponding Toolkit.Objective 3.3: Evaluate the effectiveness of CIL cadre as measured through the impact of professional learning on behavior.
Project Methods
The project will progress in 4 stages, beginning with system- and district- wide data walkthroughs and ending with evaluations of the Train-the-Trainer program and Toolkit, as well as the impact of both on CIL behaviors. The project will rely on Senior Key Personnel, the Advisory Board, and DoDEA Leadership to provide feedback and guidance throughout 4 stages of project development. Stage 1 of this project will use a data-based decision-making process and data walk throughs to address the DoDEA's capacity for change throughout the system. Stage 2 of this process, the initial Modules for a Train-the-Trainers program and Toolkit, will focus on the knowledge, skills, and tools that system-level stakeholders (i.e., the CIL cadre) need to acquire about continuous improvement, changing school culture, and leadership. Stage 3 of the process will involve the design, implementation, and evaluation of inquiry-focused Modules that the CIL cadre can use to address change at the level of classroom practice. Finally, Stage 4 of the project will involve the design, implementation, and evaluation of the Modules and Toolkit for the CIL cadre, focusing on the resources for professional learning design that will allow change to become job-embedded and sustainable throughout the DoDEA system. By the end of Stage 4, the DoDEA system will be positioned to move forward with Phase 2 of its continuous improvement process: Establish Strategic Objectives and Initiatives.We will use a backwards design process to design the Train-the-Trainers Program, which will begin by aligning each of the module topics with the appropriate DoDEA and AdvanceEd Standards. Based on these standards, we will create a comprehensive Train-the-Trainers assessment plan that includes the standards-based formative and summative assessments for the program. The sequence of content and activities in each module will follow the prepare, inquire, act, evaluate continuous improvement cycle. The Project Team and coaches will work closely with the Advisory Board and DoDEA Leadership Team to solicit feedback about the design, content, and activities of each module.The Train-the-Trainers program will be implemented using a hybrid program delivery model, comprised of a web-based instructional system and face-to-face activities, to deliver a series of educational modules. Implicit in each Train-the-Trainer module is a continuous cycle of preparing for change, inquiring into instructional practices, acting on the analysis of evidence, and evaluating actions. This inquiry-based approach to each of the module topics will facilitate a process-oriented mindset towards continuous improvement amongst the CIL cadre. To facilitate our capacity to meet project goals, the Project Team will rely on continuous feedback from the DoDEA Leadership Team, Project Team Advisory Board, and locally-relevant data sources that pertain to college and career readiness and STEM integration.Each Train-the-Trainer module will be taught and assessed using a project-based approach. Participants will select data-based problems related to their practices, and apply the preparing, inquiry, action, and evaluation techniques that they learn in the module to the problem. Sample problems that program participants may address in their data based projects include: developing school capacity to address College and Career Ready Standards or integrating STEM activities across the general education curriculum. Experts in instructional leadership and continuous improvement will be available for online coaching with participants throughout the semester as they develop projects. Additionally, the modules will draw from locally-relevant evidence related to college and career readiness and STEM integration to facilitate professional learning of these topics. The modules will rely on whole group, small group, and independent activities. The activities will be based on the best principles of professional development for adult learners. We will operationalize the modules using job-embedded, collaborative approaches to professional learning, like book studies, instructional rounds, lesson studies, and peer coaching groups. Using these models will facilitate the CIL cadre's capacity to later implement these same models with principals and teachers in schools.The Project Team, in conjunction with the Advisory Board and DoDEA Leadership, will also develop a Toolkit for use in the continuous development module. This Toolkit will contain all of the resources necessary to implement the Train-the-Trainer modules. It will also include a resource guide that lists references, websites, research projects, and links to organizations focused on college and career readiness, STEM integration, military families in schools, school improvement, and professional learning design and evaluation. Experts from our Advisory Board and expert coaches will assist in development of this Toolkit.Project objectives will be assessed by an independent evaluation team, the American Institute of Higher Education (AmHighEd), LLC. The AmHighEd team will apply their past experience with design and assessment of total quality systems; auditing quality systems for quality awards and accreditations; assessment and measurement of various processes within academia and industry; evaluation of research articles and research projects; preparation of strategic plans and evaluation of vision, mission and strategic goals of organizations; and advisement of management teams for continuous improvement.The evaluation conducted by AmHighEd will consist of three major components.Component 1 : Systemic AlignmentItem 1. Verify the gaps and opportunities for improvement identified through the Self-Assessment and Data Analysis step with the stakeholders.Item 2. Validate metrics and measures selected as Targets for Improvements with stakeholders and organization's overall strategic plans.Item 3. Evaluate the effectiveness of corrective actions and preventive actions identified during implementation.Component 2 - Evaluation of the Train-the-Trainer program and corresponding Toolkit Item 1. Validate needs diagnosed through interviews or data collection with stakeholders.Item 2. Evaluate alignment of objectives with diagnosed needs.Item 3. Evaluate completeness of learning plans to satisfy the defined objectives. o Item 4. Verify and validate the evaluation methods.Component 3 - Evaluation of the Effectiveness of CIL as Measured through the Impact of Professional Learning tem 1. In addition to evaluating items such as content and trainer proficiency, the evaluator will evaluate behaviors.Item 2. Evaluate the effectiveness of the analysis methods with respect to the pre-determined objectives.Item 3. Evaluate the effectiveness of the improvement plans over time (e.g., examine trends, feedback loops, and validation with stakeholders).

Progress 09/01/16 to 08/31/17

Outputs
Target Audience:The professional learning programs involved employees of the Department of Defense Education Activity (DoDEA), including all of the personnel for the Centers for Instructional Leadership (n=21), Directors for Student Excellence (n=3), District Superintendents (n=8), and Community Superintendents (n=18), as well as many personnel from DoDEA Headquarters (n=51). In total, we delivered programming to 101 different individuals. Changes/Problems:As a cooperative agreement, this project has involved ongoing, close collaboration with DoDEA, often negotiating fine details to ensure the project activities are aligned with the immediate and emerging needs of the stakeholders. On a broader scale, an example of a negotiated investment is the creation and refinement of a Learning Walkthrough Tool and Guide that will be applied DoDEA-wide beginning in January 2018. The idea for a common Walkthrough emerged through Virginia Tech's facilitation of strategic planning during the leadership symposium in September 2016, and the ongoing work extended through professional learning activities facilitated during the leadership symposium in June 2016. Another example of changes to anticipated program delivery is the inclusion of DoDEA Headquarters (HQ) personnel in many of the professional learning modules. While HQ was not originally identified as a target audience, we added a three-hour face-to-face workshop delivered exclusively for HQ personnel, and we facilitated two 90-minute webinars for DoDEA's Assessment Task Group, which was composed primarily of HQ personnel. These changes have helped to promote a common language and a broader approach to continuous school improvement than was originally articulated. Beyond adjustments to plans for delivery of professional learning, there were also significant changes to plans for evaluation. As we learned more about the evaluation needs associated with the project and the relationship with the expertise of the external evaluation group (AmHighEd), it became apparent the original team and approach was going to be insufficient. We terminated our contract with AmHighEd and requested a no-cost extension on the project to allow additional time for the comprehensive CIL evaluation report. In some ways, the extended time may align with best practice, because DoDEA is still working to fully staff the CILs, which in turn dictates the need for more time to generate and share artifacts for consideration in the evaluation. What opportunities for training and professional development has the project provided? Training and professional development is directly aligned with the project goal two, which involves facilitation of train-the-trainer modules in virtual and in-person delivery formats. Highlights of professional learning offerings include: Three sessions on data-based decision-making and strategic planning for 2016 DoDEA Leadership Symposium at the Joint Expeditionary Base Little Creek-Fort Story, Virginia, with 38 participants. Two sessions on strategic planning with DoDEA CIL Chiefs (and a few headquarters personnel) in December 2016. Three, three-hour webinars for Module 1, "The Role of Professional Learning Communities in Continuous Improvement," with 64 participants. Three, three-hour webinars and one in-person workshop for Module 2, "Developing Capacity with Professional Learning Communities," with 69 participants. Three, three-hour webinars for Module 3, "Implementing Change with Professional Learning Communities," with 68 participants. Three, five-day, on-site Professional Learning Institutes on "Instructional Leadership Skills for Continuous Improvement," with 48 participants. Two 90-minute assessment webinars with DoDEA Assessment Task Group in May 2017. Multiple sessions on continuous improvement skills for instructional leadership for the 2017 DoDEA Leadership Symposium at Fort Benning, Georgia, with 67 participants. How have the results been disseminated to communities of interest?As a cooperative agreement, this project has involved ongoing, close collaboration with DoDEA, often negotiating fine details to ensure the project activities are aligned with the immediate and emerging needs of the communities of interest. One change to anticipated program delivery was the inclusion of DoDEA Headquarters (HQ) personnel in many of the professional learning modules. While HQ was not originally identified as a target audience, we added a three-hour face-to-face workshop delivered exclusively for HQ personnel, and we facilitated two 90-minute webinars for DoDEA's Assessment Task Group, composed primarily of HQ personnel. These changes have helped to promote a common language and a broader approach to continuous school improvement for all of DoDEA's communities of interest. When providing reports to the DoDEA Project Lead, we have typically provided both full reports and summary reports, with the idea that the summary reports are appropriate for sharing within DoDEA among communities of interests. Examples of these reports include: Summary Report of Post-Session Assessments for Modules 1-3 [Report Date: May 18, 2017] Summary Report of Post-Session Assessments for Spring 2017 Professional Learning Institute [Report Date: May 18, 2017] Summary Report of DoDEA CIL Professional Development Needs Assessment [Report Date: June 9, 2017] Insights on Progress from DoDEA Leadership Advisory Board (LAB) for the Centers for Instructional Leadership (CILs) [Report Date: July 18, 2017] Summary Report on Insights from Focus Groups with DoDEA's Centers for Instructional Leadership [Report Date: August 20, 2017] What do you plan to do during the next reporting period to accomplish the goals?During the next reporting period, we will do the following to accomplish the phase one project goals: 1. Continue established communication processes with DoDEA. 2. Confirm final deliverables and process the necessary project re-budget. 3. Finalize structure for Instructional Leadership Toolkit and ensure all requested tools from phase one of the project are reflected in the Toolkit. 4. Finalize project evaluation components of phase one.

Impacts
What was accomplished under these goals? Within the major goals of the project, we can identify key accomplishments to date: System-wide, standards-based, collaborative model for continuous school improvement. Conveyed a structure and support for guiding improvement efforts, including: Presenting framework to serve as DoDEA's Cycle for Improvement (Figure 2). Facilitating leadership symposium sessions on "Continuous Improvement, Theories of Action, and Strategic Planning"; "Data Walkthroughs and Root-Cause Analysis;" and "Continuous Improvement Skills for Instructional Leadership." Facilitating small group meetings on "Planning for Success"; "An Appreciative Approach to Improvement"; "Assessment Inventory: Optimizing the Portfolio of Purposes"; and "Assessment-Related Ideas that Everyone Almost Certainly Already Knows but Might Benefit from Discussing." Providing templates and protocols for theories of action, root cause analysis, strategic planning, action planning, and marketing through DoDEA-wide CIL newsletter. Convening Leadership Advisory Board (LAB) meetings to facilitate broad involvement in planning processes. Provided comprehensive support for development of a Learning Walkthrough Tool and Guide, including: Sharing tools and resources to guide a common approach. Facilitating internal reviews of preliminary drafts of the Tool and Guide. Facilitating leadership symposium sessions on "Learning About the Learning Walkthrough: Components and Implementation." Delivering a complete 44-page PDF guide, including job aids and sample reports. Train-the-Trainer program and Continuous Improvement Toolkit to use with the CIL cadre districts, and schools. Provided structure and guidance for practical approaches for professional learning by shaping professional learning content with the following: Train-the-trainer curriculum maps, instructional guide template; professional learning needs assessment; protocol for feedback loops; and professional learning evaluation instrument. Facilitated train-the-trainer modules in virtual and in-person delivery formats, including: Three, three-hour webinars for Module 1, "The Role of Professional Learning Communities in Continuous Improvement," with 64 participants. Three, three-hour webinars and one in-person workshop for Module 2, "Developing Capacity with Professional Learning Communities," with 69 participants. Three, three-hour webinars for Module 3, "Implementing Change with Professional Learning Communities," with 68 participants. Three, five-day, on-site Professional Learning Institutes on "Instructional Leadership Skills for Continuous Improvement," with 48 participants. Provided a framework and initial content for DoDEA's Continuous Improvement Toolkit to be presented and maintained via Schoology. Comprehensive CIL evaluation report. While the comprehensive CIL evaluation report is forthcoming, we have already prepared components, and the data has been insightful for guiding the ongoing work: Key considerations emerged from discussions with DoDEA CIL LAB: Findings highlight early evidence of changes occurring as a result of professional development activities and investment of time and energy of CIL personnel. Investment to build awareness and trust with instructional leaders has both immediate and long-term benefits for establishing a solid foundation for learning and leading continuous improvement. Formal and informal networks are starting to emerge as a result of professional development, expanding communication channels and building new ones. Systematic data-informed processes are emerging and contributing to decisions about professional learning. The alignment of core values and desired results are having an impact on the daily work of instructional leaders as DoDEA builds the capacity for the "One DoDEA" vision. The module essential questions yielding the most change in knowledge were: How can CIL leadership turn around module 1 and support the embedded work associated with collaborative practices in a way that shifts the culture from one of teaching, to one that is about student learning? [0.9 change related to module 1] Why are protocols in the PLC process important, and when is it appropriate to use them? [0.8 change related to module 2] What opportunities do learning walkthroughs present to facilitate collaboration between schools, districts, and CILs? [0.7 change related to module 4e] How do the 3 big ideas of PLC and the 4 critical questions of a team drive the work of collaboration? [0.7 change related to module 1] How can the CIL help school leaders to leverage knowledge about the change process? [0.7 change related to module 4a] The Professional Development Needs Assessment revealed CIL priorities: Respondents appreciate focused and timely professional learning experiences. The competencies with highest priority for professional learning are in the areas of "Professional Development" and "Facilitative Leadership Skills." The most recent data reveals the highest priority competency is "Implement Professional Learning Experiences," followed by "Cognitive Coaching." Collaborate closely with DoDEA to achieve all project goals. We engaged in post-award kick-off meetings to clarify project deliverables, and we have worked to adapt as new considerations emerged. We convened weekly check-in meetings with DoDEA Project Lead, discussing project related activities, based on a consensus agenda. We have used the existing DoDEA LMS structure (i.e. Schoology) as the platform for sharing project-related resources and are in the process of ensuring the organization and final form of these resources are meeting the needs of DoDEA. We worked with DoDEA to convene three meetings of a DoDEA Leadership Advisory Board, which provided input related to the project deliverables. We fulfilled DoDEA Project Lead requests related to delivery dates for participant programs, travel and security procedures, points of contact at project program delivery sites, and coordination of travel arrangements.

Publications