Source: STARRMATICA LEARNING SYSTEMS, INC. submitted to
STEM LESSON CREATION COACHING FRAMEWORK TO IMPROVE THE PEDAGOGY OF K-5 TEACHERS IMPLEMENTING THE NGSS
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
SMALL BUSINESS GRANT
Reporting Frequency
Annual
Accession No.
1024527
Grant No.
2020-39412-33215
Cumulative Award Amt.
$600,000.00
Proposal No.
2020-06673
Multistate No.
(N/A)
Project Start Date
Sep 1, 2020
Project End Date
Aug 31, 2022
Grant Year
2020
Program Code
[8.6]- Rural & Community Development
Project Director
Starr, E.
Recipient Organization
STARRMATICA LEARNING SYSTEMS, INC.
3409 VALLEY OAKS DR
CLINTON,IA 52732
Performing Department
(N/A)
Non Technical Summary
A highly effective teacher in elementary school is essential to a student's future success in STEM fields--yet many rural elementary teachers report feeling unprepared and uncomfortable teaching science. Research shows they often lack adequate content knowledge and have low confidence in their science instruction. The new demands set by the Next Generation Science Standards (NGSS) compound these issues, as their student performance expectations are set high and integrate three dimensions of learning. They are much different from previous science standards that were mere statements of knowledge. Research has specifically documented the vast gap between the way science is currently taught and the strategies promoted by the NGSS. Research shows that extensive and effective professional development (PD) is required to successfully implement the NGSS. Unfortunately, rural schools have limited PD opportunities because of financial and geographic limitations. Based on StarrMatica's Phase I R&D proof-of-concept success, the Phase II SBIR development of StarrMatica NextGen Science (SNGS) is expected to provide a unique, comprehensive online PD solution for implementing NGSS in rural districts on tight curriculum budgets and with limited PD time. This innovative framework is a "virtual science coach" designed to improve K-5 science teacher pedagogy and to facilitate effective NGSS lesson development, regardless of a teacher's previous training, experience, or curriculum materials. The Phase I goal was to show SNGS framework feasibility for more-effectively supporting a teacher's ability to create high-quality supplemental science lessons. We met/exceeded each Phase I objective by showing that elementary teachers could create a high-quality lesson with supplementary materials in much less time and with greater NGSS alignment than a lesson created with their own methods/materials. Phase I research also revealed that even though teachers were able to develop lessons with SNGS that were fully aligned to the NGSS, their instruction during those lessons was only slightly better-aligned with the methods the NGSS require. Thus, this 2-year Phase II project will be focused on further developing our platform. It will include research-based PD to improve science instruction via 1) Videos, texts, and quizzes to improve questioning strategies and NGSS knowledge; 2) 72 informational texts to improve science content knowledge for each K-5 NGSS Performance Expectation (PE); and 3) 72 lesson-planning templates organized by the 5E's with examples and exemplars for each PE. In Year 1, six teachers will participate in a software evaluation usability study. In Year 2, data will be gathered from 27 teachers in a validation study to demonstrate a teacher's a) improved NGSS understanding, b) increased science content knowledge, c) improved questioning strategies to support inquiry-based learning, and d) increased confidence in teaching NGSS-aligned science. Data will be collected via video-lesson observations, interviews, surveys, and quizzes before and after use of the framework and will be analyzed and compared to determine change or gain in scores to evaluate Phase II success.SNGS is expected to help rural elementary teachers meet the NGSS by a) addressing the key problems they face with science instruction; b) overcoming the cost/geographic limitations of PD; c) facilitating the instructional shift teachers must make to meet the NGSS by improving teacher science pedagogy; d) providing comprehensive content that supports lesson development for every K-5 NGSS PE; and e) being comprised of research-based, proven components. Our initial target market will be K-5 school districts in the 20 NGSS-adopting states, as they are expected to be seeking implementation assistance at the administrative level. However, this does not limit our potential market. The framework for NGSS coaching developed here--in particular, content that improves questioning skills and science content knowledge--will have universal application regardless of a state's NGSS adoption status. We anticipate adapting its use to other subject matter as well.
Animal Health Component
(N/A)
Research Effort Categories
Basic
(N/A)
Applied
(N/A)
Developmental
(N/A)
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
90374103020100%
Knowledge Area
903 - Communication, Education, and Information Delivery;

Subject Of Investigation
7410 - General technology;

Field Of Science
3020 - Education;
Goals / Objectives
The overall goalof this SBIR Phase II project is to fully develop, validate, and commercialize an innovative online professional development framework designed to help K-5 teachers improve their science instruction with a "virtual science coach" regardless of their previous training, experience, or curriculum materials. We will do so by improving teachers' science content knowledge, NGSS knowledge, and questioning skills as well as supporting their science lesson planning for each NGSS Performance Expectation throughout the school year. Phase II research will consist of the following Technical Objectives: Objective 1: Demonstrate the usability and effectiveness of the SNGS framework to improve a K-5 teacher's understanding of the NGSS framework. The first performance metric for this objective will be significantly higher scores on the New Framework for Science Education: Survey of Teacher Understanding (Nollmeyer & Bangert, 2017), which assesses in-service educators' self-reported understanding of the NGSS framework, for surveys taken before using SNGS (pre) versus surveys taken after using SNGS (post). The second performance metric will be significantly higher scores on StarrMatica written multiple choice quizzes within Module 1 taken after completing the NGSS professional development in Module 1 versus quizzes taken before completing the Module I professional development. Dr. Ray Reichenberg from the University of Nebraska Lincoln will compare pre- and post-surveys and quizzes using a Wald test. This test will be conducted within a structural equation modeling framework using full-information maximum likelihood (FIML) estimation. The use of FIML estimation will allow for the handling of any missing data, should that occur, assuming the data are missing completely at random (MCAR) or missing at random (MAR). The test accounts for the paired nature of the data via the estimated correlation between the means of the two variables (pre and post scores).Objective 2: Demonstrate the usability and effectiveness of the SNGS framework to improve a K-5 teacher's science content knowledge. The first performance metric for this objective will be a statistically significant increase (α = .05) in scores on The Science Beliefs Test (Larrabee, Stein, & Barman, 2006), which assesses elementary teachers' beliefs about general science content, for tests taken before using SNGS versus tests taken after using SNGS. The second performance metric will be a statistically significant increase (α = .05) in scores on StarrMatica written multiple choice quizzes within Module 2 taken after reading science content knowledge informational texts for each K-5 NGSS Performance Expectation versus quizzes taken before reading the texts. Dr. Reichenberg will compare pre- and post-tests and quizzes using a Wald test as described in Objective 1.Objective 3: Demonstrate the usability and effectiveness of the SNGS framework to improve a teacher's questioning strategies to support inquiry-based learning. The performance metric for this objective will be the ability to generate significantly more questions that are student-centered and which require a higher level of cognition in science lessons taught after using SNGS versus before using SNGS. This will be determined by having participating teachers submit one videotaped lesson from before and after their SNGS training and use. These lessons will be transcribed and scored using an internally developed question evaluation tool. Teacher questions will be coded as teacher-centered or student-centered and identified by their cognitive level according to Bloom's Revised Taxonomy (Anderson & Krathwohl, 2001). The evaluations will be completed by Dr. Dana Atwood-Blaine, PI Emily Starr, Dr. Marcy Berger, and two University of Northern Iowa STEM Graduate Students to ensure interrater reliability. Dr. Reichenberg will compare pre- and post-evaluations using means difference testing methods appropriate for proportions (i.e., the pre/post difference in proportion of questions coded as student/teacher-focused). Differences in the median pre/post cognitive level of the questions will be assessed using a method appropriate for paired, ordinal data, such as the Wilcoxon signed-rank test or the sign test for paired data (depending on the distribution of the difference scores).Objective 4: Demonstrate the usability and effectiveness of the SNGS framework to improve a K-5 teacher's confidence in his or her ability to teach NGSS-aligned science content.The performance metric for this objective will be significantly higher scores on the Science Teaching Efficacy Belief Instrument (Riggs & Enochs, 1990), which measures science teachingself-efficacy, for surveys administered before using SNGS versus surveys administered after using SNGS. Dr. Reichenberg will compare pre/post surveys using a Wald test as described in Objective 1.Objective 5: Assess the impacts of StarrMatica NextGen Science on the socio-economic development of rural areas This project is intended to help reduce the STEM educational opportunity inequities between rural and urban school districts by supporting more effective STEM (science) elementary instruction through StarrMatica's SNGS virtual framework. For this Phase II project, StarrMatica will focus on two categories of socio-economic impacts: 1) Socio-economic impacts to rural areas beyond the project period; and 2) Economic impacts accruing to students beyond the project period due to improved academic outcomes and income potential, and to teachers due to improved STEM strategies and science content knowledge.
Project Methods
In Year 1, StarrMatica will first develop the virtual science coaching framework and then test its usability with six elementary teachers, one from each grade level K-5. In Year 2, twenty-seven elementary teachers will participate in a validation study.Year One: Usability StudySix elementary teachers have agreed to participate in the Year 1 usability study,Teachers will complete an online training session to familiarize themselves with the SNGS framework. They will then be asked to: 1) Complete the NGSS framework and questioning professional development including quizzes; and 2) Choose one performance expectation template. Then read the science content knowledge informational text and take the quiz, and use the template to modify or develop one science lesson. Upon completion, these teachers will respond to an online questionnaire that asks about their experiences using the platform. The focus of the questionnaire will be how well the framework improved the teachers' NGSS knowledge, science content knowledge, and questioning skills, and how well the framework supported the teachers' lesson development. Ease of use and functionality will also be evaluated. Graduate assistants will analyze the questionnaireit through Qualtrics.Individual interviews will be conducted with each teacher to obtain more detailed information about how the teachers used SNGS, what features worked well and what elements can be improved to provide the highest quality PD for K-5 teachers.The six teachers who participate in the usability study will not be participating in the validation study.Year Two : Validation StudyBaseline Date GatheringAfter completion of the usability study and final adjustments to the software, StarrMatica NextGen Science will be deployed to twenty-seven elementary teachers that have agreed to participate in the Year 2 validation study (see facilities list and letters of participation.) Before accessing SNGS, baseline data will be gathered for each objective. For Objective 1, teachers will complete the New Framework for Science Education: Survey of Teacher Understanding, which assesses in-service educators' self-reported understanding of the NGSS framework. For Objective 2, teachers will complete The Science Beliefs Test, which assesses elementary teachers' beliefs about general science content. For Objective 3, teachers will video record themselves teaching a science lesson to students. These "pre-lessons" will be transcribed by a transcription service and then evaluated using an internally developed question evaluation tool. For Objective 4, teachers will complete the Science Teaching Efficacy Belief Instrument, which measures science teachingself-efficacy.The New Framework for Science Education: Survey of Teacher Understanding, The Science Beliefs Test, and the Science Teaching Efficacy Belief Instrument are research-validated instruments. The internally developed question evaluation tool will be created by Dr. Atwood-Blaine, PI Emily Starr, and Dr. Marcy Berger based on the research of Oliveira (2010). Oliveira defines three categories of student-centered oral questions (referential, clarification, and confirmation) and two categories of teacher-centered oral questions (display and comprehension). The study provides descriptions and examples for each category that will be used to guide evaluators. Based on the research that teachers must ask higher level questions for students to deeply understand concepts (Orlich, 1980; Smith et al, 1993, Wilen and Clegg. 1986; Yet et. al, 1998), the evaluation tool also incorporates a coding system to assign each question a cognitive level according to Bloom's Revised Taxonomy, which is ahierarchical classification system that ranks the cognitive processes that students use.Framework Training and UseAfter the initial baseline data has been collected, the twenty-seven teachers will complete an online training sessionto familiarize themselves with the SNGS framework. They will then be asked to accomplish four tasks in this order: 1) Complete the NGSS framework and questioning professional development including taking the "pre" and "post" quizzes; 2) Use the NGSS performance expectation templates as they modify and develop science lessons throughout the year; 3) Read or listen to the audio of all of the science content knowledge informational texts for their grade level and complete the related pre- and post-quizzes, and 4) Video record themselves teaching a science lesson to students that was planned using an SNGS performance expectation template. These post-lesson videos will be transcribed by a transcription service.The internally developed question evaluationtool will be used to code the transcribed lessons.Post-Use Data GatheringAt the end of the research period, teachers will be asked to once again complete the New Framework for Science Education: Survey of Teacher Understanding, The Science Beliefs Test, and the Science Teaching Efficacy Belief Instrument. In addition, research teachers will complete the same feedback questionnaire deployed in the Year 1 usability study and will participate in an informal interview.Data Analysis For Objective 1, wewill assess the effectiveness of the SNGS framework to improve a K-5 teacher's understanding of the NGSS framework. To do so, wewill compare pre- and post-data from the New Framework for Science Education: Survey of Teacher Understanding and the multiple choice quizzes accompanying the NGSS framework PD in Module 1 using a Wald test method capable of handling any missing data, should that be present. For Objective 2, to assess the effectiveness of the SNGS framework to improve a K-5 teacher's science content knowledge, wewill compare pre- and post-data from The Science Beliefs Test and the multiple choice quizzes accompanying the science content knowledge informational texts using the paired samples t-test approach. For Objective 3, to assess the effectiveness of the SNGS framework to improve a teacher's questioning strategies to support inquiry-based learning, wewill compare pre- and post-lesson questioning evaluations using a method appropriate for assessing median differences in paired, ordinal data, such as the Wilcoxon signed-rank test or the sign test for paired data (depending on the distribution of the difference scores). For Objective 4, to assess the effectiveness of the SNGS framework to improve a K-5 teacher's confidence in his/her ability to teach NGSS-aligned science content, wewill compare pre/post data from the Science Teaching Efficacy Belief Instrument using the Wald test method cited previously. Finally, data within each objective will be grouped by grade level and analyzed to determine if any grade level(s) showed significantly more improvement in any Objective 1-4. This analysis will determine to what extent SNGS successfully addresses the pedagogical differences between K-5 grade levels.For Objective 5, StarrMatica will focus on socio-economic impacts to rural areas and to students and teachers beyond the project period. We will calculate impact ranges based upon reasonable estimates of product adoption, existing research-based estimates (both educational surveys and econometric studies of human capital development) of instructional impacts on student outcomes and standardized test scores, student outcomes and school quality measures on rural retention and in-migration, the effects of accumulated human capital on rural incomes, and individual income expectations relative to teacher qualifications and student academic outcomes. Estimates and data will be collected from StarrMatica's Phase II validity study. Estimates/projections will be calculated utilizing projections, information, and data from academic literature in education and human capital development, and income and industry multiplier data from the U.S. Bureau of Economic Analysis.

Progress 09/01/20 to 08/31/22

Outputs
Target Audience:StarrMatica's Virtual Science Coach enables an innovative online approach to science professional development for K-5 teachers thateliminates the cost of missed instructional time in the classroom, is sustained over a period of time in order to effect change, canbe implemented by a district thus avoiding costly live trainers, andsetsup long-term assistance for all current and future teacher in a district.This unique combination of benefits, combined with our research-based content that focuseson teaching pedagogy, will provide a competitive advantage in the marketplace. There are approximately 122,550 public and private elementary schools in the country and 171 DOD schools around the world, housing ~1.3 million classrooms. StarrMatica will initially target administrators in the 20 states that have adopted the Next Generation Science Standards (NGSS) followed by the 24 states whose standards are based on the NGSS. This represents a total addressable market of ~910,200 teachers or $728,160,000 at our current per teacher price. Elementary teachers in NGSS-adopting states are a secondary market, both because they are seeking out ways to align their instruction with the NGSS and because they need professional development hours for licensure renewal. A survey by The New Teacher Project found that school districts pay an average of $18,000 per teacher per year on professional development (TNTP, 2015). The 50 largest school districts in the country spend ~$8 billon on professional development services annually. This figure is supported by further survey data which reveals that teachers spend an average of 19 school days, or almost 10% of a typical school year, in professional development training (TNTP, 2015). Yet, 69% of teachers report receiving no science coaching and an additional 24% report PD only 1-3 times per year. Our product's online delivery and ability to improve both lesson content and instructional pedagogy regardless of a schools' chosen curriculum will make it particularly relevant for meeting the needs of rural school districts who struggle with curriculum resources, budgets, and teacher retention. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?The Virtual Science Coach provided the opportunity for eight research teachers (K-5)to participate in onlinescience professional development over the course of the 2021-2022 school year. How have the results been disseminated to communities of interest? Nothing Reported What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? Impact Statement 77% of our workforce requires STEM skills. But only 30% of our college students choose STEM careers. The consequences of this shortage will negatively impact our economy; however, as educators who care deeply about every student, we know the consequences of an ineffective STEM education are far more drastic. Students will have a significantly narrowed job market, reduced income potential, and fewer opportunities to contribute their unique ideas to society changing advancements. The STEM workforce gap is created in elementary school. By 8th grade, almost half of students are no longer interested in STEM - and of those still interested, there are half as many girls. Research shows that the most effective way to solve this issue is to spark an interest in STEM in elementary school. Yet reading and math, not science, are the priority in K-5 classrooms. In fact, science is one of the first things to be sacrificed because teachers are so overwhelmed. Teaching science seems complex - standards are complicated, curriculum materials are sparse, and time with students is limited. To solve this problem,our Phase II effort is reinventing professional development with "bite sized" coaching that empowers busy classroom teachers to become highly effective science educators. Instead of ineffective, expensive, and time-consuming live workshops or online classes, The Virtual Science Coach's (VSC) online sessions - each 20 minutes or less - intently focus on research-based teaching strategies proven to raise student achievement in science. While lesson plans describe WHAT to teach, our method shows teachers HOW to teach science. Without this foundational science teaching knowledge, research shows that even the most high-quality lesson plans do not significantly improve student achievement. The VSC was designed by university professors and experienced classroom educators to ensure that no matter how much time is devoted to science or what lesson materials are available, teachers understand how spark an interest in STEM. Detailed Accomplishments Per Objective: StarrMatica's Virtual Science Coach (VSC) solves the problem of ineffective K-5 science instruction through innovation in its professional development content and delivery model. Phase II research validates that the VSC is an effective method for improving elementary teacher science instruction regardless of experience or curriculum materials. All five research objectives were met. The VSC improved a K-5 teacher's understanding of the NGSS as evidenced by statistically significant differences between pre- and post-test scores measuring their knowledge of the NGSS. The difference was large in magnitude and translated to an increase from the 50th to the 95th - 99th percentile. The VSC improved a K-5 teacher's science content knowledge as evidenced by post quiz scores that were significantly higher than their pre scores, and this difference was large in magnitude (r = .72). Post scores on the Science Beliefs Test were not significantly higher, and this is hypothesized to be a result of the shortcomings of the assessment and its appropriateness for the study rather than an indication of a lack of effectiveness of the VSC. Rather than completing all of the texts for their grade level in the Performance Expectations Coach, teachers chose one text to complete for our research. The text they chose correlated to only one or two of the 49 items on the Science Beliefs Test. So, participants could not be expected to improve their test scores since the information on the test was not addressed in their chosen VSC materials. Furthermore, quizzes that were directly aligned to the material each teacher read resulted in significantly higher post scores, indicating an increase in knowledge. Once a teacher has read all of the texts for their grade level, we hypothesize the Science Beliefs Test would be a more effective measurement tool. The VSC improved a K-5 teacher's questioning strategies in three areas: productive questions, productive talk moves, and asking higher level thinking questions as evidenced by transcript analysis of pre- and post video lessons. The median proportion of all questions asked that were productive was .21 before completion of the VSC platform and .47 after completion. Teachers' post scores were not significantly higher than their pre scores, but the difference was large in magnitude (r = .70). The median number of productive talk moves was 6 before completion of the platform and 22 after the platform. This difference was statistically significant and large in magnitude (r = .80). The median question cognitive level for understanding (the second level in Bloom's Taxonomy) was 8 before completion of the platform and 30 after completion. This difference was statistically significant and large in magnitude (r = .81). There were few questions asked at Bloom's levels higher than understanding, so this has been identified as an area for additional VSC content in the future. Teachers also completed a pre- and post quiz aligned to each questioning video. The median proportion of questions answered correctly was .77 before completion of Module 2 videos and .97 after completion. Teachers' post scores were significantly higher than their pre scores, and this difference was large in magnitude (r = .90). The VSC improved a K-5 teacher's confidence in his or her ability to teach NGSS-aligned science content. This was evidenced by an increase in scores on the Science Teaching Efficacy Beliefs Instrument. Teachers' post-use efficacy beliefs were significantly higher on average than their pre-use efficacy beliefs, and this difference was large in magnitude (0.89 standard deviations or an increase from the 50th percentile to the 81st percentile). The VSC is expected to have an impact on the socioeconomic development of rural areas in the following ways: a) Exposure to a highly effective teacher, like those trained by the VSC to be effective in teaching science, results in an income increase of ~$7,000 by age 28. In addition, as discussed previously, students who are interested in STEM in elementary are more likely to pursue STEM careers which earn twice the salary of non-STEM occupations. b) A single school building can expect to save between $4,724 and $6,345 in professional development costs and teacher turnover costs because of their VSC implementation. In addition to the proven effectiveness of its content, the Virtual Science Coach's innovative online delivery model overcomes the time and resource obstacles faced by school district leaders - particularly in rural areas. "Bite-sized coaching" focuses on research-based teaching strategies that that help teachers understand HOW to teach science no matter their given curriculum or how much time is available for instruction. Online delivery ensures access no matter where a school is located, and eliminates expensive travel, substitutes, lost instructional time, and live trainers. VSC sessions require 20 minutes or less which ensures steady progress and can be completed during district professional development time, grade level planning time, lesson planning time, or personal learning community meetings. A step-by-step plan allows easy in-district implementation and ensures fidelity across buildings. The VSC provides a sustained professional development solution so all current and future teachers in a district have foundational science teaching knowledge and an understanding of district expectations.

Publications


    Progress 09/01/20 to 08/31/21

    Outputs
    Target Audience:StarrMatica's Virtual Science Coach enables an innovative online approach to science professional development for K-5 teachers thateliminates the cost of missed instructional time in the classroom, is sustained over a period of time in order to effect change, canbe implemented by a district thus avoiding costly live trainers, andsetsup long-term assistance for all current and future teacher in a district.This unique combination of benefits, combined with our research-based content that focuseson teaching pedagogy, will provide a competitive advantage in the marketplace. There are approximately 122,550 public and private elementary schools in the country and 171 DOD schools around the world, housing ~1.3 million classrooms. StarrMatica will initially target administrators in the 20 states that have adopted the Next Generation Science Standards (NGSS) followed by the 24 states whose standards are based on the NGSS. This represents a total addressable market of ~910,200 teachers or $728,160,000 at our current per teacher price. Elementary teachers in NGSS-adopting states are a secondary market, both because they are seeking out ways to align their instruction with the NGSS and because they need professional development hours for licensure renewal. A survey by The New Teacher Project found that school districts pay an average of $18,000 per teacherper yearon professional development (TNTP, 2015). The 50 largest school districts in the country spend ~$8 billon on professional development services annually. This figure is supported by further survey data which reveals that teachers spend an average of 19 school days, oralmost 10% of a typical school year, in professional development training (TNTP, 2015). Yet, 69% of teachers report receiving no science coaching and an additional 24% report PD only 1-3 times per year. Our product's online delivery and ability to improve both lesson content and instructional pedagogy regardless of a schools' chosen curriculum will make it particularly relevant for meeting the needs of rural school districts who struggle with curriculum resources, budgets, and teacher retention. Changes/Problems:Research Note: The University of Nebraska-Lincolnis subcontracted to conduct the research analysis for this award. In December 2021, Dr. Ray Reichenberg left the University, and the project was subsequently reassigned to Dr. Natalie Koziol. What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest? Nothing Reported What do you plan to do during the next reporting period to accomplish the goals?Objective 1 Plans for the next reporting period: During the next reporting period, the eleven research teachers will complete their post Survey of Teacher Understanding. The data from the pre and post Survey of Teacher Understanding as well as the pre and post NGSS Coach quizzes will be analyzed by Dr. Natalie Koziol of the University of Nebraska-Lincoln according to the analysis plan detailed in this objective. The teachers will also complete a post-use questionnaire and interview, the purpose of which is to gain feedback about the platform for future development and marketing. Objective 2 Plans for the next reporting period: During the next reporting period, the eleven research teachers will each choose at least one performance expectation they will be teaching to their students. The teachers will read or listen to the content knowledge text for that performance expectation and will complete the related pre and post quiz. Objective 3 Plans for the next reporting period: During the next reporting period, the eleven research teachers will teach their post-lesson and upload the video of their instruction. Both pre and post lessons will be transcribed and scored. The data from the pre and post lessons as well as the pre and post questioning video quizzes will be analyzed by Dr. Natalie Koziol of the University of Nebraska according to the analysis plan detailed in this objective. Objective 4 Plans for the next reporting period: The eleven research teachers will complete their post Science Teaching Efficacy Belief Instrument. The data from the pre and post Science Teaching Efficacy Belief Instrument will be analyzed by Dr. Natalie Koziol of the University of Nebraska according to the analysis plan detailed in this objective. Objective 5? Plans for the next reporting period: During their post-lesson questionnaire, teachers will be asked: 1) Would you recommend that your district purchase StarrMatica's Virtual Science Coach? Why or Why Not? and 2) If your district provided professional development on demand for you like StarrMatica's Virtual Science Coach when implementing new curriculum (such as the NGSS) would you be less likely to leave that district and look for a teaching position elsewhere? Why or why not? Mark will consult the data collected from all pre and post assessments completed by research teachers. During the next reporting period, Mark Imerman will use the questionnaire and research data to look for 1) Socio-economic impacts to rural areas beyond the project period; and 2) Economic impacts accruing to students beyond the project period due to improved academic outcomes and income potential, and to teachers due to improved reading and science instructional strategies. He will determine this by calculating impact ranges based upon reasonable estimates of product adoption, existing research-based estimates (both educational surveys and econometric studies of human capital development) of instructional impacts on student outcomes and standardized test scores, student outcomes and school quality measures on rural retention and in-migration, the effects of accumulated human capital on rural incomes, and individual income expectations relative to teacher qualifications and student academic outcomes.

    Impacts
    What was accomplished under these goals? Impact Statement: 77% of our workforce requires STEM skills. But only 30% of our college students choose STEM careers. The consequences of this shortage will negatively impact our economy; however, as educators who care deeply about every student, we know the consequences of an ineffective STEM education are far more drastic. Students will have a significantly narrowed job market, reduced income potential, and fewer opportunities to contribute their unique ideas to society changing advancements. The STEM workforce gap is created in elementary school. By 8th grade, almost half of students are no longer interested in STEM - and of those still interested, there are half as many girls. Research shows that the most effective way to solve this issue is to spark an interest in STEM in elementary school. Yet reading and math, not science, are the priority in K-5 classrooms. In fact, science is one of the first things to be sacrificed because teachers are so overwhelmed. Teaching science seems complex - standards are complicated, curriculum materials are sparse, and time with students is limited. To help, our Phase II effort is reinventing professional development with "bite sized" coaching that empowers busy classroom teachers to become highly effective science educators. Instead of ineffective, expensive, and time-consuming live workshops or online classes, The Virtual Science Coach's (VSC) online sessions - each 20 minutes or less - intently focus on research-based teaching strategies proven to raise student achievement in science. While lesson plans describe WHAT to teach, our method shows teachers HOW to teach science. Without this foundational science teaching knowledge, research shows that even the most high-quality lesson plans do not significantly improve student achievement. The VSC was designed by university professors and experienced classroom educators to ensure that no matter how much time is devoted to science or what lesson materials are available, teachers understand how spark an interest in STEM. Accomplishments to Date Executive Summary: Development has been completed on the StarrMatica Virtual Science Coach platform which includes professional development components focused on the Next Generation Science Standards and Questioning skills. In September, four usability study teachers used the platform and completed a feedback questionnaire. In October and November, eleven research study teachers completed three initial assessments as well as a baseline questionnaire. They also filmed themselves teaching a science "pre" lesson. Currently, research teachers are using the Virtual Science Coachmodules to improve their questioning skills, standards knowledge, and science content knowledge. During the next reporting period, teachers will be choosing a performance expectation they would like to be the focus of their science "post" lesson. They will read or listen to the associated content knowledge text, and then they will use the lesson planning template to guide their lesson development. Finally, teachers will video tape their execution of that lesson. In addition, teachers will complete their three "post" assessments along with an end-of-study questionnaire and interview. The data from all assessments, quizzes, and video lessons will be analyzed by Dr. Natalie Koziol of the University of Nebraska according to the analysis plan detailed in our five objectives. Detailed Accomplishments for Each Objective: Objective 1 Research Completed to Date: Four usability study teachers completed the pre quizzes and watched the videos in September. Feedback was collected via a post-use questionnaire and Zoom interview by the principal investigator. The questionnaire focused on the usability and functionality of StarrMatica's Virtual Science Coach. Sample questions included: Module 1 contained information about the NGSS. What part of this module did you find the most useful? Why? What part of this module did you find the least useful? Why? What changes would you recommend to strengthen Module 1? What additional features/functions would you like added to the platform? How would you rate the overall usability of the platform? In October and November, eleven research teachers completed the New Framework for Science Education: Survey of Teacher Understanding (Nollmeyer & Bangert, 2.017), which assesses in-service educators' self-reported understanding of the NGSS framework. In addition, research teachers completed a baseline questionnaire that asked about their current science teaching practices. Research teachers have also completed all pre and post quizzes and watched all videos in the NGSS Coach section of the platform. Results: The questionnaire feedback from the four usability study teachers was overwhelmingly positive. Teachers felt they had learned from the videos and would recommend them to their colleagues. Any changes suggested were minor and aesthetic such as including a graphic with each "coach" section on the teacher's dashboard. The research teachers' NGSS Coach pre and post quiz data and baseline questionnaire results have not yet been exported and analyzed. The Survey of Teacher Understanding data has also not yet been analyzed since the post test has not yet been given. Objective 2 Research Completed to Date: In October and November, eleven research teachers completed the The Science Beliefs Test (Larrabee, Stein, & Barman, 2006), which assesses elementary teachers' beliefs about general science content. Research teachers are in the process of completing the pre and post quizzes and reading or listening to the texts for their grade level. Results: The Science Beliefs Test data has not yet been analyzed since the post test has not yet been given. Pre and post quiz data has not yet been exported and analyzed because not all teachers have completed the Performance Expectations Coach section. Objective 3 Research Completed to Date: Four usability study teachers completed the pre quizzes and watched the videos in September. Feedback was collected via a post-use questionnaire and Zoom interview by the principal investigator. In October and November, eleven research teachers uploaded their pre-lesson videos. Research teachers have also completed all pre and post quizzes and watched all videos in the Questioning Coach section of the platform. Teachers are currently planning and teaching their post lessons using the lesson planning template. Results: The feedback from the four usability study teachers was overwhelmingly positive. Teachers felt they had learned from the videos and would recommend them to their colleagues. The pre-lesson videos have not yet been coded and analyzed. Both the pre and post lessons will be sent to the transcription company and then scored by researchers together to help ensure consistency of scoring. The Questioning Coach pre and post quiz data has not yet been exported and analyzed. Objective 4 Research Completed to Date: In October and November, eleven research teachers completed the Science Teaching Efficacy Belief Instrument (Riggs & Enochs, 1990), which measures science teachingself-efficacy. Results: The Science Teaching Efficacy Belief Instrument data has not yet been analyzed since the post test has not yet been given. Objective 5 Research Completed to Date: Mark Imerman, the socio-economic consultant for this project, has completed his external research of academic literature in education and human capital development, and income and industry multiplier data from the U.S. Bureau of Economic Analysis.

    Publications