Progress 07/01/19 to 02/28/21

Outputs
Target Audience:The addressable market for the product as a result of this project is elementary and middle school administrators and teachers. There are ~22.5 million students enrolled in 3rd through 8th grade public schools and charter schools in the United States in ~1.3 million classrooms. Private schools and Department of Defense schools include an additional ~1.5 million students and ~200,000 classrooms. StarrMatica's initial target market is rural school districts, with over 9.1 million students and ~916,000 classrooms. Purchasing decisions in the education market are typically made by a team of decision-makers, including the purchaser and end-users. The purchaser of our product will be a curriculum director at the district level, a principal at the building level, or a teacher at the classroom level. The end-users will be classroom teachers. StarrMatica's initial target market is rural school districts, with over 9.1 million students and ~916,000 classrooms. We will expand that market to include elementary and middle schools across the U.S., given that only 40% of fourth graders and 34% of eighth graders nationwide are proficient in math, as measured by the 2018 National Assessment of Educational Progress. The primary drivers in the target market are the need improve math performance and to provide personalized learning. The top K-12 technology spending priorities on the 2017-2018 Center for Digital Education survey were: 1) personalized learning and 2) digital content and curriculum. And 96% of districts responding shared that their district "provides software or digital curriculum to classrooms specifically to encourage personalized learning practices" (Center for Digital Education, 2018). Secondary drivers include the proliferation of devices in the classroom and the increasing demand for digital content as an instructional tool. 93% of public schools have a ratio of devices to students that is 1:2, 1:1, or even 2:1 (Maylahn, 2017). As the number of devices in schools increase, the already significant demand for digital instructional content will also increase. Digital content accounted for 56.8% of all K-12 instructional materials purchased in 2016, a significant increase from the previous year, and the first year to be over 50% (SIMBA Information, 2017). A national emphasis on STEM education is also expected to be a significant driver in the market. States are increasingly introducing legislation and offering grants focused on improving STEM education (Solano, 2017) Changes/Problems:Three fourth grade teachers from a rural school district participated in the Math Your Way effectiveness study. Two were research classrooms and the third was a control classroom. Before the pandemic, research participants included 3rd, 4th, and 5th grade teachers from two rural school districts. Because of the pandemic, all but two teachers from one school district declined to participate in the study. Attempts were made to recruit additional participants by working with the Iowa Council of Teachers of Mathematics and the Iowa STEM Council to solicit participation, by posting in various elementary math Facebook communities, and by sending email communication to our internal list of 5,000 educator contacts. Our four-month recruitment efforts produced no additional research participants. Teacher data was collected through pre- and post-surveys, interviews, and observations. Because the study was conducted during the global Covid-19 pandemic, in-person classroom observations were only carried out before the use of Math Your Way. What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest? Nothing Reported What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? Improving STEM performance is of economic importance to the nation as a whole. Researchers estimate that if the U.S. became a top-performing nation on international assessments between 2005 and 2025, by 2037 its GDP would be 5 percent higher and 36 percent higher by 2080 than would be the case if the U.S. remains mediocre in math and science. STEM instruction in the United States, as measured by standardized tests, is inadequate. Improvement in the National Assessment of Educational Progress (NAEP) mathematics scores has slowed in the past decade. From 1996 to 2005, average scores increased by 14 points for 4th graders and nine points for 8th graders. From 2005 to 2015, average scores only increased two points for 4th graders and three points for 8th graders. The average 2015 NAEP mathematics score actually declined by two points for 4th graders and three points for 8th graders compared to 2013, and the 2017 scores showed no improvement. These are the first declines since 1990 for fourth and eighth graders. Research has discovered a rapid decline in positive attitudes towards STEM between ages 10 and 14 This is precisely the age range when algebraic reasoning is developed - and success with algebra strongly influences students' future decisions about careers in. In fact, eighth grade is the deadline by which students must understand relevant math concepts if they are going to be successful in college and career - high school interventions to improve individual math achievement are too late. Improving high school course rigor is not effective unless students have been prepared in upper elementary and middle school mathematics to benefit from advanced courses Studies also show disengagement in mathematics during middle school has a negative impact on overall student achievement. Because data was gathered from two research classrooms and one control classroom within one small rural district, the results of the study were largely inconclusive. On the final math task assessment, students in the research classroom out performed the control classroom by eight percentage points. This data suggests the simulation increased the mathematical understanding of some of the students in the research classroom. While the prototype differentiation is similar in scope to the teachers' core curriculum materials, the presence of an audio read aloud support for all text in the prototype was a unique differentiator. For more conclusive results, further study will need to be conducted on a larger scale with additional data collecting methods that control for multiple variables not considered during this Phase I project. Our Phase I project focused on five technical objectives. Objective #1 Results Summary: To examine the effectiveness of Math Your Way to allow mathematics teachers to improve disciplinary literacy instruction, researchers developed a Mathematics Disciplinary Literacy Rubric to compare students who used the platform versus students in a control classroom. A t-test comparison of scores from students in the Math Your Way and in control classrooms was used. This study looked at three categories of disciplinary literacy comprehension: metacognition, determining importance and inferring. The students were asked to solve a math task and then discuss their process. The control group performed at a higher level in two of the disciplinary literacy comprehension skill areas: inferring and metacognition. The experimental group performed at a higher level in the skill area of inferencing. One explanation for the results could be that the experimental group did not access the disciplinary literacy instructional supports that were provided in the form of hints because students had the choice whether or not to use the hint supports. For this study, metacognition hints were only accessed 26% of the time, determining importance hints were only accessed 30% of the time, and inferencing hints were only accessed 36% of the time. Without accessing the hints, students had the opportunity to answer questions that were formulated to target those three skills but did not learn strategies that could help them to answer those types of questions. Objective #2 Results Summary: An EQuIP rubric, measuring CCSS alignment and lesson quality, was used to compare differentiation during problem solving instruction by teachers using Math Your Way in contrast to their previous methods. Researchers also used a general qualitative approach to analyze survey results, interview transcripts, and lesson observation video transcripts to look for the amount and types of accommodations provided to students with and without the use of Math Your Way. Researchers first measured a structured lesson plan from McGraw Hill, the main curriculum resource for the participating school, as compared to the Math Your Way prototype lesson. Researchers scored both lessons overall at a rating of "2," but the specific criteria for the category of instructional supports showed a higher rating with the use of the Math Your Way platform. One differentiation factor that Math Your Way provided that was not matched in the research classroom's core curriculum was the audio voiceover supports. Objective #3 Results Summary: Researchers compared student performance in control and experimental classrooms: some solving a mathematics problem solving task after using the platform as compared to others solving the same task without using the platform. Results showed 2 out of 16 students in the control group had the correct answer, and 6 out of 28 students in the classrooms using the Math Your Way platform had the correct answer. Objective #4 Results Summary: Researchers used a t-test of scores on a researcher-developed STEM Interest Inventory to compare the level of interest in STEM between students in classrooms who used the platform and students in classrooms that did not use the platform. The STEM Interest Inventory can also be analyzed based on interest, curiosity and value. Of the 18 items on the inventory, 11 measure interest in STEM, 2 measure curiosity about STEM, and 5 measure the value children recognize for STEM. For the Interest in STEM items 46% of the experimental students and 49% of the control students responded with high or highest curiosity. For the Value of STEM category, 50% of the experimental students and 56% of the control students responded with high or highest curiosity. For the Curiosity about STEM category, 23% of the experimental students and 25% of the control students responded with high or highest curiosity. Overall, there was not much difference between the control and experimental groups. Question 6: I enjoy learning engineering did show a higher degree of positive interest for the experimental group. Question 12: In school, thinking about topics like energy and matter makes me yawn elicited a higher degree of negative response for the experimental group. Objective #5 Results Summary: In order to pursue the existence of positive impacts upon students, the researchers surveyed classroom teachers both before and after using Math Your Way in the classroom. They also surveyed students in experimental classes (those exposed to Math Your Way) and a control group to determine if students in experimental classrooms were more interested and engaged with STEM skills and career possibilities than the control group. The trials conducted were insufficient to show positive economic impacts of the SBIR development grant for StarrMatica Math Your Way software. The data is indeterminate. While no positive economic impact can be estimated, we cannot discount the possibility that such benefits may exist. Longer trials with larger samples and more detailed measurement would be necessary to reach a conclusion. These conditions were not attainable during the 2020-2021 school year.

Publications

Progress 07/01/19 to 06/30/20

Outputs
Target Audience:The addressable market for the product as a result of this project is elementary and middle school administrators and teachers. There are ~22.5 million students enrolled in 3rdthrough 8thgrade public schools and charter schools in the United States in ~1.3 million classrooms. Private schools and Department of Defense schools include an additional ~1.5 million students and ~200,000 classrooms. StarrMatica's initial target market is rural school districts, with over 9.1 million students and ~916,000 classrooms. Purchasing decisions in the education market are typically made by a team of decision-makers, including the purchaser and end-users. The purchaser of our product will be a curriculum director at the district level, a principal at the building level, or a teacher at the classroom level. The end-users will be classroom teachers. StarrMatica's initial target market is rural school districts, with over 9.1 million students and ~916,000 classrooms. We will expand that market to include elementary and middle schools across the U.S., given that only 40% of fourth graders and 34% of eighth graders nationwide are proficient in math, as measured by the 2018 National Assessment of Educational Progress. The primary drivers in the target market are the need improve math performance and to provide personalized learning. The top K-12 technology spending priorities on the 2017-2018 Center for Digital Education survey were: 1) personalized learning and 2) digital content and curriculum. And 96% of districts responding shared that their district "providessoftware or digital curriculumto classrooms specifically to encourage personalized learning practices" (Center for Digital Education, 2018). Secondary drivers include the proliferation of devices in the classroom and the increasing demand for digital content as an instructional tool. 93% of public schools have a ratio of devices to students that is 1:2, 1:1, or even 2:1 (Maylahn, 2017). As the number of devices in schools increase, the already significant demand for digital instructional content will also increase. Digital content accounted for 56.8% of all K-12 instructional materials purchased in 2016, a significant increase from the previous year, and the first year to be over 50% (SIMBA Information, 2017). A national emphasis on STEM education is also expected to be a significant driver in the market. States are increasingly introducing legislation and offering grants focused on improving STEM education (Solano, 2017) Changes/Problems:Three fourth grade teachers from a rural school district are participating in theMath Your Wayeffectiveness study. Two are research classrooms and the third is a control classroom. Before the pandemic, research participants included 3rd, 4th, and 5thgrade teachers from two rural school districts. Because of the pandemic, all but two teachers from one school district declined to participate in the study. Attempts were made to recruit additional participants by working with the Iowa Council of Teachers of Mathematics and the Iowa STEM Council to solicit participation, by posting in various elementary math Facebook communities, and by sending email communication to our internal list of 5,000 educator contacts. Our four-month recruitment efforts have produced no additional research participants. Because the study is being conducted during the global Covid-19 pandemic, in-person classroom observations were only carried out before the use of Math Your Way. What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest? Nothing Reported What do you plan to do during the next reporting period to accomplish the goals?Our Phase I project focused on five technical objectives. Objective #1 Before Next Reporting Period: Students will use the prototype. Then, they will be asked to solve a math task and discuss their process. This discussion will be analyzed for the use of disciplinary literacy skills. Objective #2 Before Next Reporting Period: The remaining sections of the simulation will be storyboarded and programmed. Researchers will analyze the completed prototype using the EQuIP rubric. Objective #3 Before Next Reporting Period: Students in the research classroom will use the prototype and complete the math task. Students in the control classroom will complete the math task. Researchers will analyze the data. Objective #4 Before Next Reporting Period: Research classroom and control classroom students will complete the STEM Interest Inventory, and the data will be analyzed. Objective #5 Before Next Reporting Period: The economic analysis will be completed after all student data has been collected and analyzed.

Impacts
What was accomplished under these goals? STEM-related jobs are expected to grow at 17% compared to 9.8% for non-STEM jobs (Langdon et al., 2011). It is essential we increase the number of professionals entering STEM careers for our country's economic growth, global competitiveness, and continued innovation (Langdon et al., 2011; Bottoms & Uhn, 2007; Freeman, 2005; Jeffers et al., 2004) Yet the U.S. is trending in the opposite direction. Thirty years ago, ~40% of the world's scientists and engineers resided in the U.S. Today that number has shrunk to ~15% (Adkins, 2012). Projections indicate that the United States will experience a shortage of engineers, perhaps needing as many as one million more than our education system will produce at its current rate (President's Council of Advisors on Science and Technology (U.S.), & United States, 2012). Improving STEM performance is of economic importance to the nation as a whole. Researchers estimate that if the U.S. became a top-performing nation on international assessments between 2005 and 2025, by 2037 its GDP would be 5 percent higher and 36 percent higher by 2080 than would be the case if the U.S. remains mediocre in math and science (Hanushek & Woessmann, 2008). STEM instruction in the United States, as measured by standardized tests, is inadequate. Improvement in the National Assessment of Educational Progress (NAEP) mathematics scores has slowed in the past decade. From 1996 to 2005, average scores increased by 14 points for 4th graders and nine points for 8th graders. From 2005 to 2015, average scores only increased two points for 4th graders and three points for 8th graders (National Science Board, 2018). The average 2015 NAEP mathematics score actually declined by two points for 4th graders and three points for 8th graders compared to 2013, and the 2017 scores showed no improvement. These are the first declines since 1990 for fourth and eighth graders (National Science Board, 2018). When compared globally, fourth graders' average math scores were 11th out of 48 countries on the 2015 Trends in International Mathematics and Science Study (TIMMS), and 15-year olds (typically 9th graders) placed 38th out of 71 countries on the Program for International Student Assessment (PISA) (National Science Board, 2018). This notable decline in international performance from elementary to middle school highlights the need to improve mathematics instruction specifically between fourth and eighth grade. Research has discovered a rapid decline in positive attitudes towards STEM between ages 10 and 14 (Lyons, 2006; Renninger & Hidi, 2011; Osborne, Simon, & Collins, 2003; Simpson & Steve, 1990). This is precisely the age range when algebraic reasoning is developed - and success with algebra strongly influences students' future decisions about careers in STEM (Adelman, 2006). In fact, eighth grade is the deadline by which students must understand relevant math concepts if they are going to be successful in college and career - high school interventions to improve individual math achievement are too late (ACT, 2008). Improving high school course rigor is not effective unless students have been prepared in upper elementary and middle school mathematics to benefit from advanced courses (ACT, 2008) Studies also show disengagement in mathematics during middle school has a negative impact on overall student achievement (Doig, 2005). Our Phase I project focused on five technical objectives. Objective #1 Progress Data Gathered: The two research classroom teachers completed a baseline questionnaire. That data revealed that Teacher 1 had participated in some disciplinary literacy training through the Iowa Area Education Agency and integrated some informational texts into her math instruction. Teacher 2 had received no disciplinary literacy instruction and responded that she never uses informational texts during math instruction. Prototype Progress: Two informational texts have been written that highlight the career of robotics (the career focus of the simulation) and the real life uses of rescue robots. Three disciplinary literacy pieces have been written about rescue robots along with accompanying quiz questions that focus on the disciplinary literacy skills of metacognition, determining importance, and inferring. Objective #2 Progress Data Gathered: Teacher interviews and pre-lesson observations have been completed. Researchers analyzed teachers' pre-lessons using the EQuIP rubric. Prototype Progress: Four of the five segments of the rescue robotics simulation have been storyboarded. Three of the five have been programmed. Objective #3 Progress The math task to be used for the student task has been identified and additional disciplinary literacy questions written. Objective #4 Progress The STEM Interest Inventory has been modified for our study. Objective #5 Progress The economic consultant has completed his literature review.

Publications