Source: STARRMATICA LEARNING SYSTEMS, INC. submitted to NRP
STEM LESSON CREATION COACHING FRAMEWORK TO IMPROVE THE PEDAGOGY OF K-5 TEACHERS IMPLEMENTING THE NEXT GENERATION SCIENCE STANDARDS (NGSS)
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
Reporting Frequency
Annual
Accession No.
1015865
Grant No.
2018-33610-28268
Cumulative Award Amt.
$100,000.00
Proposal No.
2018-00872
Multistate No.
(N/A)
Project Start Date
Aug 1, 2018
Project End Date
Jul 31, 2019
Grant Year
2018
Program Code
[8.6]- Rural & Community Development
Recipient Organization
STARRMATICA LEARNING SYSTEMS, INC.
3409 VALLEY OAKS DR
CLINTON,IA 52732
Performing Department
(N/A)
Non Technical Summary
High-quality schools positively influence the economic development prospects in rural areas by attracting business and industry to the area and producing the educated labor force that is needed to sustain economic growth. The greatest source of a highly educated population in rural areas is native individuals who attended college locally or who return home after completing their education elsewhere. To guarantee that students have opportunities to enter STEM careers in rural communities, their schools must provide a high-quality education that includes foundational STEM concepts. An effective teacher is the most important factor in a student's academic success, yet issues including teacher shortages and high turnover rates affect teacher quality in rural areas. Sparking interest in science in elementary school is more effective in creating the motivation to enter STEM career paths than are later efforts to encourage high school students to take more-advanced courses. Improving science education from the outset by more effectively supporting elementary teachers in STEM instruction can have lasting effects on the preparedness of rural workforces to meet the employment demands of technology firms that rural communities are seeking to attract.The overall goal of this multi-phase SBIR project is to develop, validate, and commercialize a unique, cost-effective virtual coaching framework to improve a teacher's ability to create high-quality supplemental science lessons resulting in more-effective elementary (K-5) science instruction aligned with the Next Generation Science Standards (NGSS). The innovation of the StarrMatica NextGen Science program is its focus on cultivating effective STEM education by improving science instruction for all elementary teachers, regardless of previous training or experience. The key differentiator of this technology is the application of the known effective strategy of coaching in a digital environment through a "virtual science coach" framework. Components of the framework--including a content examples library, content evaluation tools, a research-based instructional model, lesson-self assessment tools, student science notebook interface, and data-recording features--work together to create a comprehensive science instruction improvement solution that is economical, time-saving, and replicable in every school. A pilot test and usability study, followed by a feasibility study for both pre-service and in-service teachers, will provide qualitative and quantitative data before and after use of the System. Data will be analyzed and compared to determine change or gain in scores.The release of the Next Generation Science Standards (NGSS) in 2013 compounded the already numerous issues that elementary teachers have with teaching science effectively. Teachers can't use their current aging science curriculums to meet the NGSS; however, fewer than 10% of schools plan to purchase a new science core curriculum, so teachers will need to learn to use supplementary materials to meet the standards. SNGS is expected to help teachers meet the NGSS by a) addressing the significant problems elementary teachers face with science instruction; b) overcoming the professional development geographic limitations of rural schools; c) providing a consistency of curricular materials in high-turnover rural schools from year to year and teacher to teacher; d) facilitating the instructional shift teachers must make to meet the NGSS; e) providing a comprehensive lesson-creation framework that coaches teachers through each portion of the lesson planning process in a way that improves teacher pedagogy; and f) being comprised of components that are designed based on research. Our research will show that SNGS is easy to use for both novice and experienced teachers and assists them to effectively develop high-quality science lessons. The primary addressable market is elementary administrators and teachers who must modify curricular materials to improve teacher pedagogy and to meet the NGSS standards.
Animal Health Component
(N/A)
Research Effort Categories
Basic
(N/A)
Applied
(N/A)
Developmental
100%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
90374103020100%
Goals / Objectives
The overall goal of this multi-phase SBIR development project is to develop, validate, and commercialize a unique, cost-effective virtual coaching framework to significantly improve the quality of elementary science education in rural school districts. We will do so by supporting teachers' capabilities to efficiently and effectively create high-quality supplemental science lessons even when they do not have experience with STEM concepts.Phase I research will consist of the following Technical Objectives: Objective 1: Demonstrate the feasibility of developing a virtual science coaching framework (content examples library, 5E's instructional model, lesson self-evaluation rubric, resource evaluation checklist, and professional development materials) for K-5 elementary teachers. This framework, called StarrMatica NextGen Science, will enable teachers to efficiently create supplemental lessons to meet all three "dimensions of learning" for one NGSS in less time than their current lesson preparation methods. The performance metrics for this Objective will be higher scores on the EQuIP rubric (which measures NGSS alignment and lesson quality), when evaluated by consultant Lorenza Della Donna, for lessons developed with access to SNGS versus lessons developed without access. This will demonstrate that lessons developed with SNGS have greater NGSS alignment. Educator surveys will be deployed before and after using SNGS to demonstrate that using the framework results in reduced planning time.Objective 2: Demonstrate the feasibility of the SNGS framework developed in Objective 1 to enable teachers to create a high-quality NGSS-aligned lesson with supplementary materials more effectively than teachers' current lesson preparation methods. The performance metric for this Objective will be higher scores on the EQuIP rubric (which measures NGSS alignment and lesson quality), when evaluated by Lorenza Della Donna, for lessons developed with access to SNGS versus lessons developed without access. This will demonstrate the lessons developed with SNGS are of higher quality than those developed without. Qualitative data will be obtained from video and anecdotal records of lessons observed by researcher Dana Atwood Blaine with and without access to SNGS using the Science Teacher Inquiry Rubric. This data will show evidence of greater NGSS-aligned inquiry-based teaching and learning during units developed with access to SNGS in contrast to those units developed without access.Objective 3: Evaluate the potential socio-economic impacts of this project on rural communities. This project is intended to help reduce the STEM educational opportunity inequities between rural and urban school districts by supporting more effective STEM (science) elementary instruction through StarrMatica's SNGS virtual framework. For this Phase I project, StarrMatica will focus on two categories of socio-economic impacts: 1) Socio-economic impacts to rural areas beyond the project period; and 2) Economic impacts accruing to students beyond the project period due to improved academic outcomes and income potential, and to teachers due to improved STEM strategies and science content knowledge.
Project Methods
To demonstrate the feasibility of Objective 1 and Objective 2, StarrMatica will first develop SNGS materials, then carry out a pilot study with three components: 1) The first study will use NGSS's EQuIP rubric to compare science units developed by 12 pre-service elementary teachers at the University of Northern Iowa, half of who will use SNGS and half who will not. 2) The second study will focus on obtaining feedback from Central Springs and Easton Valley in-service teachers and their students on their perceptions of SNGS in comparison to the supplementary science resources and lesson creation methods they normally use. 3) The third study will deploy SNGS in six elementary classrooms in Central Springs and Easton Valley. The EQuIP rubric will be used to evaluate science lessons developed by experienced teachers before being introduced to SNGS and then again after receiving access to SNGS. To demonstrate the feasibility of Objective 3 (socio-economic impacts), data from the studies will be evaluated by our economist consultant to determine the potential socio-economic impacts of this project on rural communities.Objective 1 and 2: SNGS Feasibility StudyClassroom Studies Part 1: Effectiveness for Novice Teachers. To ensure that SNGS is an easy and effective tool for even inexperienced teachers to use, the system will be tested on pre-service elementary teachers in a highly controlled environment. As part of their normal coursework, and after 12 weeks of instruction focused on the NGSS and the 5E learning cycle instructional model, ~12 pre-service teachers enrolled in an Elementary Science Methods course at Iowa's premier teacher preparation college, University of Northern Iowa, will be asked to develop a 5-lesson science unit that aligns with a particular NGSS standard and follows the 5E instructional model. Half of the students (i.e., the experimental group) will be given access to SNGS and instructed to use the tool to help them develop their units. The other half of the students (i.e., the control group) will develop their units without access to SNGS. Twelve of the units (six from each group) will be evaluated by Lorenza Della Donna using the "Educators Evaluating the Quality of Instructional Products" (EQuIP) Rubric to determine how well they meet the intended goals of the NGSS standards. The EQuIP rubric for science provides criteria by which to measure the alignment and overall quality of lessons and units with respect to the NGSS. Means for the two groups will be statistically compared using an independent sample t-test to determine if the units created by students using SNGS score significantly higher on the EQuIP rubric than those created by students who do not use SNGS.Classroom Studies Part 2: Pilot-testing and Usability Feedback. A mixed-methods approach will be used to collect usability feedback from a group of six in-service elementary teachers and their students. Six third-grade teachers will pilot-test SNGS and provide feedback to the developers. Teachers in the pilot test will complete a pre-survey that asks them about the supplemental resources and methods they normally use to develop their science lessons, and the amount of time they spend on lesson development. These teachers will then receive one day of professional development to support their use of SNGS. Pilot-test teachers will use SNGS to supplement their core science curriculum throughout one unit. These teachers will be provided a digital log in which to record a description of their use of SNGS and to provide feedback about the system. Participating teachers will also be provided professional support from the StarrMatica team throughout the pilot-test period. At the conclusion of the unit, pilot-test teachers will complete a post-survey to provide feedback about using SNGS and how it compares to other supplementary science resources and methods they normally use, and the amount of time they typically spend on lesson development. Both the pre- and post-surveys will include Likert-style items that ask about teachers' use and opinion of specific supplementary science resources that could be considered potential competitors with SNGS (e.g., supplementary software and libraries of open educational content) In addition, the surveys will ask qualitative, open-ended questions to allow teachers to give examples and be more specific in their feedback about SNGS and other common supplementary science materials and lesson development methods. This data will be used to help determine if Objectives 1 and 2 have been met. Feedback will also be gathered from students regarding their experiences using SNGS. Participating students will complete a post-survey, providing usability feedback for SNGS and how it compares to supplementary resources/materials typically provided by their teacher.Classroom Studies Part 3: Effectiveness for Experienced Teachers. In addition to participating in all components of the Part 2 pilot test, teachers will also participate in the Part 3 effectiveness study. Like the pre-service teachers in Part 1, these teachers will be asked to create and teach a 5-lesson mini-unit to address an assigned NGSS standard, using any available supplementary resources desired and to track time spent on lesson development. After participating in the day of professional development, these teachers will then create and teach another 5-lesson mini-unit to address an assigned NGSS standard using SNGS and to again track time spent on lesson development. Both the pre- and post- StarrMatica units will be evaluated by Lorenza Della Donna using the EQuIP rubric. Change or gain scores for each teacher will be reported and compared to the statistical difference (if any) discovered in Part 1. Time spent on lesson development will also be reported and compared. Part 3 teachers will participate in two face-to-face interviews with the researcher, once after creating the first mini-unit and once after creating the second mini-unit. The first interview will focus on the teachers' perceptions of the ease of lesson planning and effectiveness of the lessons developed without use of SNGS. The second interview will focus on teachers' perceptions of ease of lesson planning and effectiveness of lessons developed with SNGS. Finally, the researcher will conduct and record classroom observations during both the initial 5-lesson mini-unit as well as classroom implementation of the 5-lesson unit using SNGS. Video and anecdotal records of lesson observations will be analyzed qualitatively using the Science Teacher Inquiry Rubric (STIR) to identify evidence of NGSS-aligned inquiry-based teaching and learning (Bodzin & Beerer, 2003).Objective 3 will focus on evaluating the potential socio-economic impacts of this project on rural communities. As explained in the objective, StarrMatica will focus on socio-economic impacts to rural areas and to students and teachers beyond the project period. The performance metric for this objective will be the definition of impact ranges based upon reasonable estimates of product adoption, existing research-based estimates (both educational surveys and econometric studies of human capital development) of instructional impacts on student outcomes and standardized test scores, student outcomes and school quality measures on rural retention and in-migration, the effects of accumulated human capital on rural incomes, and individual income expectations relative to teacher qualifications and student academic outcomes. Estimates and data will be collected from StarrMatica's Phase I pilot feasibility study, with estimates/projections calculated utilizing projections, information, and data from academic literature in education and human capital development, and income and industry multiplier data from the U.S. Bureau of Economic Analysis.

Progress 08/01/18 to 07/31/19

Outputs
Target Audience:Our initial target market will be K-5 schools in states that have adopted the NGSS because they are expected to be actively looking for implementation assistance at the administrative level. As of October 201920states have adopted the NGSS; however, this does not limit our market. Teachers in non-adopting states are still using the NGSS standards, and strong demand exists for new science materials in non-adopting states. The framework for science instructional coaching developed for this project will have universal applications regardless of the status of a state's NGSS adoption (Cavanagh, 2016; Heitin, 2015). Purchasing decisions in the education market are typically made by a team of decision-makers, including the purchaser and end-users. The purchaser of our product will be a curriculum director at the district level or a principal at the building level. The end-users will be classroom teachers. School administrators who must defend their curriculum purchases to stakeholders want to see validated research results. This Phase I and Phase II SBIR research will provide data to show that SNGS can more effectively support all elementary teachers in creating high-quality NGSS aligned lessons and in teaching STEM subjects. The primary drivers in the target market are the need modify curricular materials and to improve teacher pedagogy to meet the NGSS standards. Secondary drivers include the proliferation of devices in the classroom and the increasing demand for digital content as an instructional tool. Public schools in the United States now provide at least one computer for every five students, and that ratio will be decreasing (Herold, 2015). In 2016, 87% of districts planned to purchase tablets, 86% of districts planned to purchase laptops, and 83% of districts expected to purchase desktop computers (Catalano et al., 2015). As the number of devices in schools increase, the already significant demand for digital instructional content will also increase. Digital instructional content is currently the largest slice of the (non-hardware) K-12 educational technology market, with annual sales of more than $3 billion, and digital content and curriculum was the second K-2 IT priority on the 2015 Center for Digital Education survey (Dunlap-Kahren, 2015). Our initial target market will be K-5 schools in states that have adopted the NGSS because they are expected to be actively looking for implementation assistance at the administrative level. As of May 2016, 18 states have adopted the NGSS; however, this does not limit our market. Teachers in non-adopting states are still using the NGSS standards, and strong demand exists for new science materials in non-adopting states. The framework for science instructional coaching developed for this project will have universal applications regardless of the status of a state's NGSS adoption (Cavanagh, 2016; Heitin, 2015). Purchasing decisions in the education market are typically made by a team of decision-makers, including the purchaser and end-users. The purchaser of our product will be a curriculum director at the district level or a principal at the building level. The end-users will be classroom teachers. School administrators who must defend their curriculum purchases to stakeholders want to see validated research results. This Phase I and Phase II SBIR research will provide data to show that SNGS can more effectively support all elementary teachers in creating high-quality NGSS aligned lessons and in teaching STEM subjects. The primary drivers in the target market are the need modify curricular materials and to improve teacher pedagogy to meet the NGSS standards. Secondary drivers include the proliferation of devices in the classroom and the increasing demand for digital content as an instructional tool. Public schools in the United States now provide at least one computer for every five students, and that ratio will be decreasing (Herold, 2015). In 2016, 87% of districts planned to purchase tablets, 86% of districts planned to purchase laptops, and 83% of districts expected to purchase desktop computers (Catalano et al., 2015). As the number of devices in schools increase, the already significant demand for digital instructional content will also increase. Digital instructional content is currently the largest slice of the (non-hardware) K-12 educational technology market, with annual sales of more than $3 billion, and digital content and curriculum was the second K-2 IT priority on the 2015 Center for Digital Education survey (Dunlap-Kahren, 2015). Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?For the project's feasibility study, the teachers were given one hour of professional development on the prototype. Additional professional development was embedded in the prototype in the form of videos and documents that taught educators about the Next Generation Science Standards and the 5Es. Each of the eight teachers was observed using the prototype with their students for one lesson. After the lesson, the teachers completed a questionnaire and participated in an interview. The professional development and subsequent use of StarrMatica Next Generation Science helped teachers to think of ways they could improve their Next Generation Science Standards instruction.This was evidenced by qualitative data from teacher questionnaires and interviews. Sample quotes include: "I love the platform. I currently use a science curriculum that is at least 15 years old. This was current, colorful, vibrant and made learning fun. I do not see anything that needs improvement. I think it is user friendly and awesome." "I loved the flow of the platform. I also lovedthe different offerings under the 5E's. It makes meeting the NGSS standards so much more attainable and sensible. When I first read the NGSS standard, I rememberthinking, "Wow, this is really powerful! If I only had the time to build lessons that could effectively meet these standards!" Thanks to the SNGS platform-- it is possible!" "I loved the StarrMatica resources! These resources are perfect for differentiation of instruction and finding ways to reach all learners. The visuals and videos really aided my ELLs, students with IEPs, and my struggling readers. The texts that supported the lessons encouraged higher order thinking for my higher level students. The vocabulary integrated within the texts while being supported by the visuals and pictures was so meaningful to my students. I loved the step-by-step guides and the options you can click to create the lessons--this really tailored the lessons to the variety of learners in my classroom. I loved the student on-line component of the program and my students loved it, too!" How have the results been disseminated to communities of interest? Nothing Reported What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? High-quality schools positively influence the economic development prospects in rural areas by attracting business and industry to the area and producing the educated labor force that is needed to sustain economic growth (Barkley and Henry, 2004). The greatest source of a highly educated population in rural areas is native individuals who attended college locally or who return home after completing their education elsewhere (Gibbs, 2005). To guarantee that students have opportunities to enter STEM careers in rural communities, their schools must provide a high-quality education that includes foundational STEM concepts. An effective teacher is the most important factor in a student's academic success, yet issues including teacher shortages and high turnover rates affect teacher quality in rural areas (Chetty, Friedman, & Rockoff, 2011; Hanushek, Kain, O'Brien, & Rivkin, 2005; Rivkin, Hanushek, & Kain, 2005). Sparking interest in science in elementary school is more effective in creating the motivation to enter STEM career paths than are later efforts to encourage high school students to take more-advanced courses (Maltese & Tai, 2016). Improving science education from the outset by more effectively supporting elementary teachers in STEM instruction can have lasting effects on the preparedness of rural workforces to meet the employment demands of technology firms that rural communities are seeking to attract. The release of the Next Generation Science Standards (NGSS) in 2013 compounded the already numerous issues that elementary teachers have with teaching science effectively. Teachers can't simply align their current curriculum materials to the NGSS and continue teaching as they have in the past. According to the 2012 National Survey of Science and Mathematics Education, 58% of elementary science classes use textbooks that are seven years old or older (Interactive Educational Systems Design, Inc., 2015). One solution is to purchase an entirely new science core curriculum--a solution that is flawed for several reasons: 1) Science concepts change quickly, so this means that it does not take long for textbooks to lose relevance (Economist, 2012); 2) schools have limited science curriculum budgets (Appleton, 2007); 3) fewer than 10% of U.S. schools plan to purchase new science core curriculum (Interactive Educational Systems Design, Inc., 2015); and 4) rural schools have even less funding for curriculum than their urban counterparts (Center for American Progress, 2011; DeNisco, 2015). This means that 90% of teachers must, by default, supplement their current (and aging) core curriculum to meet the NGSS. This is an enormous task with the complexities of the NGSS, and it is unrealistic given the lack of tools currently available. The purpose of the STEM lesson creation coaching framework (SNGS) is to help teachers meet the NGSS by a) addressing the significant problems elementary teachers face with science instruction; b) overcoming the professional development geographic limitations of rural schools; c) providing a consistency of curricular materials in high-turnover rural schools from year to year and teacher to teacher; d) facilitating the instructional shift teachers must make to meet the NGSS; e) providing a comprehensive lesson-creation framework that coaches teachers through each portion of the lesson planning process in a way that improves teacher pedagogy; and f) being comprised of components that are designed based on research. Our research showed that SNGS is easy to use for both novice and experienced teachers and assists them to effectively develop high-quality science lessons. The overall goal of the Phase I project was to demonstrate the feasibility of a virtual coaching framework to significantly improve the quality of elementary science education in rural school districts. This was to be accomplished by supporting teachers' capabilities to efficiently and effectively create high-quality supplemental science lessons even when they do not have experience with STEM concepts. Our Phase I project focused on three technical objectives. We met each one within the budget and timeline of the research period, as follows: Objective 1: Our goal was to show that we could develop a platform that would integrate a content examples library, 5E's instructional model, NGSS coaching tools, and professional development materials for K-5 elementary teachers. This platform would enable teachers of all experience levels to develop high quality NGSS - aligned lessons with reduced planning time. Results Summary: The framework was successfully created and testing confirmed the platform is functioning 100% of the time on all operating systems and devices. Qualitative survey data showed that SNGS was user friendly, supported teachers' efforts to create high quality NGSS-aligned lessons regardless of previous NGSS or curriculum development training and experience, supported teachers' efforts to create high quality NGSS-aligned lessons compared to their normal methods and materials, and supported teachers' efforts to learn more about the NGSS and the 5Es. Quantitative survey data showed that SNGS helped teachers become significantly more comfortable creating an NGSS-aligned science lesson. Teachers' self-reported comfort level with creating an NGSS-aligned lesson increased to 6.4 (on a 10-point scale, SD = 1.96); up from 4.91 on the pre-survey. Furthermore, SNGS helped teachers spend less time developing their science lessons, and the lessons they developed were better-aligned with the NGSS. Teachers reported a decrease from almost 5 hours to an average of 3 hours spent developing their science lessons. Objective 2: Our goal was to show that teachers are able to create a high-quality NGSS-aligned lesson with supplementary materials more effectively than their current lesson preparation methods. Results Summary: Data from the STIR and NGSS Lesson Screener showed the teachers were able to create a high-quality NGSS-aligned lesson with supplementary materials more effectively than their current lesson preparation methods. Teachers had an average positive change of 2.86 points on the STIR from pre-lesson to post lesson. On the NGSS lesson screener, teachers had an average positive change of 9.25 points from pre-lesson to post lesson. Every post lesson reviewed with the lesson screener received a score of 18, the highest possible score, which indicates the lessons have extensive evidence of being fully aligned with the NGSS. This is of particular note since several teachers had pre-lessons with no or inadequate NGSS alignment. Objective 3: Our goal was to show that SNGS helps to reduce the STEM educational opportunity inequities between rural and urban school districts by investigating the economic impacts accruing to students beyond the project period due to improved academic outcomes and income potential, and to teachers due to improved STEM strategies and science content knowledge. Results Summary: The rural socio-economic impacts study showed that student improvements in the development trial classrooms could range between approximately $217,800 to $399,300 in current value of lifetime expected earnings increases. Additionally, teacher turnover savings could range between $524 to $2,145 per elementary building per year for K-5 elementary schools with two classes per grade.

Publications