Source: 7 GENERATION GAMES INC submitted to
GROWING MATH: ACCESSIBLE, CROSS-CURRICULAR EDUCATION IN FOOD AND AGRICULTURAL SCIENCES
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
Reporting Frequency
Annual
Accession No.
1024845
Grant No.
2021-67037-33377
Cumulative Award Amt.
$1,000,000.00
Proposal No.
2020-09622
Multistate No.
(N/A)
Project Start Date
Nov 15, 2020
Project End Date
Nov 14, 2022
Grant Year
2021
Program Code
[A7701]- SARS-COV-2 Digital Learning Resources
Project Director
De Mars, A.
Recipient Organization
7 GENERATION GAMES INC
2111 7TH ST APT 8
SANTA MONICA,CA 904051279
Performing Department
(N/A)
Non Technical Summary
The Coronavirus pandemic has caused an urgent need in tribal and adjacent communities for new innovations to serve students learning from home - laying bare the inability of the vast majority of educational software to meet that need.Growing Mathwill provide an interactive digital platform populated with classroom-ready educational resources and teacher support tools that will enable the rapid deployment of lessons for students in Grades 3-8 that can be used in the classroom, at home or in a hybrid model. Resources can be accessed offline and on low-end hardware.With 130 distance learning modules, 123 lesson plans and leveraging existing open educational resources, Growing Math seeks not to reinvent the wheel, but to transform a wagon wheel into a performance racing tire.In a one-stop site, teachers will be able to access standards-aligned curriculum, download software, see student data, get technical support and participate in professional development.Each educational module includes a game, cultural component, mathematics component, agricultural science component and data collection, aimed at increasing math and science achievement.Growing Math can be implemented in pilot schools within the first month of funding. Funding will enable us tostreamline deployment and implementation of existing resources while significantly augmenting the library of educational modules and lesson plans.This regional project by 7 Generation Games will serve 1,500 teachers and 27,000 students across 240 schools in tribal communities and surrounding areas in six states: Arizona, Minnesota, New Mexico, North Dakota, Oregon and South Dakota.
Animal Health Component
50%
Research Effort Categories
Basic
0%
Applied
50%
Developmental
50%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
90360503020100%
Goals / Objectives
?The goals of the Growing Math project are:One, to develop and rapidly deploy lesson plans for students in grades 3-8 that can be used in the classroom, online or in a hybrid model by school in tribal communities and surrounding areas in six states; Arizona, Minnesota, New Mexico, North Dakota, Oregon and South Dakota. Lessons integrate math, food and agricultural science, Indigenous history and educational games.Second, to evaluate and scale this model to allow it to serve as a foundation for accessible, adaptive education throughout the United States.To meet these goals, the project will address the following seven objectives:Develop a web-based portal for grades 3- 8, centered around 106 existing modules aligned with math and social studies integrated with lesson plans for food and agricultural science, where resources can be accessed directly or downloaded or later, off-line use.Conduct a multi-phase rollout of curriculum, beginning December 2020, from teacher training through implementation and formative assessment with 240 schools served by the end of the 2022 academic year.Provide technical support to schools with a maximum 24-hour response time.Significantly increase student completion of mathematics and science assignments, both in-class and remote learning.Significantly increase achievement on mathematics and science standards.Create 24 new modules in response to formative evaluation data.Conduct multi-state analysis of impact on student performance and engagement.
Project Methods
Recruitment, Selection and TrainingEarly beta schools (N=15) have already been recruited in North Dakota, South Dakota and Oregon on the Spirit Lake, Standing Rock and Warm Springs reservations. Additional schools are being recruited by our partner, Native Innovation, a contracted training provider for 25 schools on the Navajo Nation. These schools were selected based on three criteria: on or adjacent to a reservation, in a rural area and commitment to begin testing as soon as funding is received. Additional schools will be recruited from the 2,800 educators on our mailing list who teach in predominantly Native American schools.Subsequent schools will be selected based on meeting one or more of the following criteria: location on or adjacent to reservation, rural community, need for curriculum accessible offline, written commitment from administrator and teachers, Title I school and serving predominantly Native American students. Schools which meet more criteria will receive higher priority. Participants who terminate the project early will be replaced first with another teacher from the same school if available. If no teacher is available, then a school from the next phase (beta, early adopter or late adopter) will be offered the opportunity to move up.Participant ActivitiesEach school will begin implementation with a two-hour hands-on teacher training, introducing site lesson plans, educational games, instructional resources (e.g. videos, virtual field trips) and technical support availability. In a follow-up session, teachers will modify their curriculum map to include Growing Math lessons, with students beginning use in the quarter in which training occurs. From January to March 2021, 15 early beta schools will use the Growing Math curriculum, with consulting teachers providing feedback on accessibility, student engagement, curriculum alignment and user experience. Early beta users will access 7 Generation Games existing teacher site. Throughout the first quarter, the site will be augmented and adapted with feedback from these teachers and their students, culminating in beta portal launch. An additional 50 beta schools will be added in the second quarter (before the 2020 school year end).Over the summer, 120 more schools will receive professional development and be prepared to implement the program in the fall semester. The first nine months of Year 1 are aimed at deployment of existing classroom-ready modules, portal development and refinement based on beta feedback (e.g. UX/UI design, types of lessons desired), ramping up services and number of schools and, simultaneously, collecting formative evaluation data on school implementation barriers and unmet needs.Based on experience, it is anticipated that the majority of schools will elect training and implementation at the beginning of the school year with other schools adapting the software mid-year in response to changing conditions, i.e., moving to a learn at home or hybrid model. Throughout the 2021-22 school year, additional individual schools and districts will be brought on board, with a full cohort of 240 schools to be included in 2022-23. The combination of monthly virtual training, software distribution by download and online access of data reports enable classrooms to come online with the project according to their own schedule. Based on feedback from educators and students, in Year 2, the project will revise educational modules to address areas of greatest needs.Teachers are busy and COVID-19 has multiplied the demands on their time. Including training and ready-made standards-aligned lesson plans with in-class, remote and blended learning options into professional development days incorporates Growing Math into the existing schedule and saves teachers time, thus increasing the probability of use. Cross-curricular education, with agricultural science incorporated in mathematics and social studies classes, will lead to more time for each subject, as students learn both together. Higher student engagement, due to learning in context, translates into higher student performance. Generally, the more time and more applications in which students learn information, the greater the retention.EVALUATIONStudent Data CollectionReal-time reports are used to monitor progress toward the project's objectives. Data will be collected within the educational games component on number of students using the software, total sessions of software usage, average sessions per student, total minutes software was used, average minutes of software usage per student, total problems completed, percentage of problems completed correctly. When students are working offline, data is saved in local storage on the device. This feature allows use in areas with unreliable or sporadic Internet. In communities where students are connecting to wifi via school bus hotspots during drop off of meals or assignments, this background data transfer can occur seamlessly as long as the application is open on the device.Teacher Data CollectionThe number of reports accessed by teachers, total instructional resources linked from class or school websites and total instructional resource views or downloads are collected through a WordPress site analytics plugin which produces daily, weekly and monthly reports. Number of teachers registered for each webinar or onsite training will be collected by the Eventbrite system used for registration. The number of teachers who attend each webinar or onsite training will be recorded by the Project Director from signup sheets. Number of teachers registered, entering bugs and number of bugs entered in Bugzilla is recorded automatically.Qualitative data will be collected through observation of training sessions, semi-structured interviews with teachers and review of digital resources produced. Each participating teacher will be paid a small stipend for a half-hour interview with the Principal Investigator. Data will be collected throughout the project with different teachers interviewed each month.Data Analysis and ReportingStatistical formative reports are generated weekly by the Project Director from the automated report systems described above, with descriptive statistics generated for all of the quantitative variables on which data is collected. These reports are used for monitoring and project management. Reports on statistics from the prior month are discussed in monthly internal meetings with all staff. Quarterly analyses in a formative evaluation report are distributed via email to all participating teachers, discussed in teacher interviews and posted on the project website. This report also includes a summary of common themes in qualitative data, a discussion of teacher retention in the project and student perseverance in software usage. The major student outcome reported is the number and percentage of students each quarter who have mastered specific Common Core mathematics, Next Generation Science Standards, state social studies standards and foundations of agricultural literacy (American Farm Bureau Foundation for Agriculture, 2020) as measured by correctly completing educational challenges within learning games. The primary teacher outcome assessed is whether they are actually implementing changes in curriculum. These regular evaluations identify both strengths and areas of concern, enabling any necessary corrective action to be taken in a timely fashion.

Progress 11/15/20 to 11/14/22

Outputs
Target Audience:Two types of audiences benefited directly from our efforts: 1,375 educators and 47,681 students. During the project period, training was received by 1,375 educators. The goal of the Growing Math project was to provide math and agricultural science integrated with Indigenous history and culture and 92% of participants in training were from six targeted states: North Dakota, South Dakota, Minnesota, Oregon, Arizona and New Mexico. This includes teachers in the small portion of the Navajo Nation that extends into Utah and teachers in Nebraska who taught students from the Pine Ridge and Rosebud reservations, which extend from South Dakota to the Nebraska border. This also includes participants who, while not residing or teaching in the target states, served students from reservations in those states, predominantly teachers in Nebraska serving students from South Dakota reservations. The remaining 8% of participants attending training were educators in programs serving predominantly Indigenous students with an interest in replicating the project in their region. The modal teacher taught multiple grades, sometimes in the same classroom, in a small, rural, predominantly Indigenous school. Nearly two-thirds (64%) of teachers taught multiple grades. This included 34% who taught students in grades K-8 or K-12, 14% who taught multiple elementary grades in the same classroom, 12% who taught multiple middle school grades and 4% who taught multiple high school grades. Of the teachers who taught a single grade level, 33% were teaching grades 3-8 and 3% grades K-2. Over 80% of participants were classroom teachers, with paraprofessionals comprising the largest group of non-teachers, followed by professors of teacher education and administrators. Although the training was available to any educator in the six target states, the majority of educators in attendance taught at schools on or adjacent to reservations. STUDENTS Students benefited both through use of teaching resources by their teachers in presenting mathematics and agricultural science content - lesson plans, activities, videos, slide presentations and demonstrations - and through hands-on use of the digital resources - educational games, instructional resources and simulations - created by the project. During the project period, 12,698 students had access to resources downloaded by their schools that were only usable offline, that is, did not require an Internet connection for use and therefore did not collect usernames. An additional 34,938 students used resources online. These are students who accessed the resources directly, tracked by username. This combined total of 45,681 is almost certainly an undercount as there were another 45,417 downloads of applications that could be used either online or offline and it is assumed that certainly some students used those offline and were not recorded in the database. Further, as documented in teacher interviews and evaluation surveys, additional students were also exposed to project resources indirectly through lesson plans downloaded and used by their teachers. While schools prohibit collection of individually identifying data on students, users are connected with their classroom teacher for data reports and thus aggregate data can be reported. The students served were overwhelmingly in grades three through eight, as noted above, in the six targeted states and disproportionately attended schools that were on reservations and majority Indigenous students. Only one teacher reported teaching at a school where students did not have devices to access the Internet (Chromebook or iPad) provided to students, in a remote community that did not have Internet access. Students disproportionately (over 90%) attended Title I schools, where at least 40% of the students are from low-income families. Students had Internet access at all but one school, and access was provided out of school for remote learning, hybrid learning and homework through either subsidized home Internet access, MiFi (personal hotspot devices provided by schools), cards loaded with credit for data download or buses with hotspots that parked in local communities for scheduled hours each day. By the end of the project period, in 2022, all students were back in school learning onsite, with occasional remote learning due to either outbreaks of COVID-19 or weather conditions that made bus transportation unsafe. While schools still provided devices to students, some programs providing Internet access had ended, resulting in a greater need for applications available offline, as indicated by the 71% increase in offline-accessible apps downloaded in 2022 compared to 2021. By contrast, our online only app usage increased only 34%, and this despite two new applications being released in 2022. Changes/Problems:Changes that occurred were not as much in scope but in order of priorities and format. A significant change in priorities occurred as a result of input from the first two cohorts of teachers. Prior to remote learning, data on student engagement and performance had not been a major issue. With students no longer "in front of them", teachers were far more interestedin data, particularly in time on task. Thus, new reports were created that showed not only the number of problems answered correctly (standards achieved) but also the number attempted and the number of sessions that students had interacted with the platform. This was positive in that it allowed more and earlier validation of data to be used to assess impact. It did, however, cause a small backlog in year one of lessons and educational resources to be added to the platform as our staff created and validated reports. In year two, more instructional resources were created, exceeding the grant objectives. Addition of an asynchronous training option was not planned, but as many of the sessions are recorded for our own staff training purposes, for review for quality improvement and uploaded to the web for teachers who attended the training toreview, much of the resources for asynchronous training were available. With high absenteeism and some schools moving back and forth among on-site, remote and hybrid learning, teachers were time-limited even more than usual. Asynchronous training was added in year two, and is maintained for new and continuing teachers who wish to use Growing Math resources. Individualized instruction for students with learning differences was not initially a major focus of the project but was frequently requested. Teaching on small, rural schools, many teachers taught multiple grades and the need for accommodations for students below grade level was an added strain on already limited resources. Recommendations for individualized instruction were added to lesson plans to accommodate students with special needs. An unanticipated positive benefit of these accommodations was an increase in usage of the curriculum in the lower grades K-2, as teachers used the video content or demonstrations portions to teach Indigenous culture and agricultural literacy content with students who were below the required level of numeracy or literacy to complete entire modules. What opportunities for training and professional development has the project provided?Fifty-four teacher training workshops were offered. These are in addition to the 22 conference presentations during the project period. How have the results been disseminated to communities of interest?In addition to one peer-reviewed article, 22 conference presentations and 54 teacher training workshops, project staff have engaged in community dissemination activities, including: Presentation at Twin Cities Start-Up Week Burns Ortiz, M., Young, S. & Wulf, M. "Too Many Children Left Behind: Improving Equity in Education through Innovation." . Sept. 23, 2021. Minneapolis. Exhibitors at Minot State University Pow-wow, Minot, ND April, 2021. Yuba City Pow-wow, Yuba City, CA , June, 2021 North Dakota Indian Education Summit, Bismarck, ND, July, 2021 United Tribes Technical College pow-wow, Bismarck, ND, September 2021 Silver City Red Paint Pow-wow, September, 2021 Cahokia Art Gallery (Native-owned art collective) , Phoenix, AZ, October, 2021 National Indian Education Association National Conference, . Oklahoma City, OK, October, 2022 National Johnson-O'Malley Association (NJOMA) Conference. September 20, 2022. Las Vegas, NV. Dissemination through electronic media included weekly newsletters sent to 2,474 subscribers, two websites (Growing Math and 7 Generation Games), social media accounts on Twitter, Facebook and linkedIn. The Growing Math project was also highlighted in several local and mass media publications and programs including a Scripps National News broadcast story on 7 Generation Game, a feature on the Fargo Forum site (InForum) and features on Minnesota Public Radio News and in Education Week. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? 1. Develop a web-based portal for grades 3- 8, aligned with math and social studies integrated with lesson plans for food and agricultural science. This portal has been created and is hosted at growingmath.org. Each educational module includes a game, cultural component, mathematics component, agricultural science component and data collection. Modules are combined by grade level in game format of 8-20 modules each. To supplement these resources the site currently contains 110 lesson plans, 305 videos and 10 miscellaneous resources (e.g., random sample simulator). This resource has been continued and extended post-USDA funding through resources from public and private sources. To date, 1,375 educators have attended training on the Growing Math platform. Lesson plans are accessed by an average of 70 teachers per day. 2. Conduct a multi-phase rollout of curriculum. Teachers from 327 schools attended training over the project period. Based on responses from the first two cohorts (N=65), in the 2020-21 academic year, a feedback form was developed and subsequent teachers had the option of submitting their feedback in writing or through interviews with the Principal Investigator or Project Director. Teachers placed an early priority on more data on student time on task and performance, since during the initial months of the project most students were learning from home. Thus, expanded reporting was moved ahead in the development schedule. After reports, the three issues most requested were; more resources for middle school, more resources for special education and pre-packaging lessons into units. In response, the project completed 26 additional modules focused on middle school standards in mathematics and science, including agricultural science. Recommendations for individualized instruction were added to lesson plans to accommodate students with special needs. One-third of the lessons have been combined into units and work is continuing post-USDA funding with support from Indigenous nations and private foundations. An asynchronous training site was created to provide access to teachers who could not attend the training held after school and on weekends. 3. Provide technical support Frequently asked questions are answered both during the online training and on the Growing Math website. Technical support can be requested using the contact form on the project website or via email to the email address provided during the Growing Math training. Both contact forms and email address are forwarded to four staff members with overlapping hours, such that the correspondence is monitored from 7am to 9pm Central time. All questions receive a response within one business day. In over 90% of the issues, one of the staff members on the response list can answer within the current business day. For more technical issues, the issue is referred to a developer for update, and the teacher kept informed of progress. 4. Significantly increase student completion of mathematics and science assignments Prior to USDA funding, an average of 7,662 students per year completed 7 Generation Games educational modules. Modules consist of instructional content, assessment and a game component.During the two-year project period, 34,938 students used resourcesonline, an average of 17,468 per year, more than double the baseline. During the two-year project, students completed 235,952 problems that were recorded in our database, an average of 6.75 per student, a significant increase over both the average of 5.3 in the five years prior to USDA funding and the 5.3 average in year one of funding. The stable level of engagement in 2020-21 was noteworthy as teachers reported, that overall student task completion of school assignments and homework had dropped dramatically during remote learning in the target schools. In the 2021-22 academic year, as students returned to the classroom and as teachers received more training on the Growing Math platform, the number of modules completed per student significantly increased. 5. Significantly increase achievement on mathematics and science standards. Data were collected for this project from January 2021 through November 2022. Baseline data was available for some standards from pre- 2020. It must be noted that there is a massive history effect that makes it impossible to conclude causality. All schools served by the project were closed for the majority of the 2020-21 academic year. Later school years were impacted by students' remote learning the prior year(s), multiple school closures and high degree of absenteeism, particularly due to the omicron variant. Data were collected on four factors; grade level of standard attempted, number of standards successfully achieved, year attempted and percentage of students achieving the standard on first attempt. Prior research documented the importance of considering item difficulty, as students may be answering problems correctly at lower grade levels at a high rate and failing at more challenging items. The number of standards mastered by students increased both from the pre-2020 baseline to 2021 and from 2021 to 2022. While students were less likely to correctly solve math challenges on the first attempt, the average number of standards mastered increased due to higher student engagement. 6. Create 24 new modules in response to formative evaluation data. This objective was exceeded with 32 new modules created. These modules included five new educational games: · Making Camp Navajo (ratio and proportion with traditional Navajo farming and crafts), · Making Camp Dakota (division word problems with traditional Dakota diet and medicinal plants), · AzTech Empiric Empire (basic statistics, fractions and basic epidemiology in the context of Latino history) - with additional funding provided by NIH, · Akiptan: Grow Your Garden (business math in the context of agricultural literacy), and · Disaster Deduction Detectives (basic statistics in the context of the impact of global warming on agriculture). Two additional applications are under development, with 10 modules completed. These applications have been supported through funding from Indigenous nations, which allows development to continue post-USDA funding. 7. Conduct multi-state analysis of impact on student performance and engagement. The impact was assessed quantitatively through analysis of student responses to math challenges and number of instructional modules completed. It was assessed qualitatively through semi-structured teacher interviews and written teacher evaluation reports. It must be emphasized that results of these analyses must be assumed to be impacted bymajorhistory effects. There was an expected decrease in the percentage of student correct answers on their first attempts at math challenges, and this decrease continued in 2022. However, the number of standards mastered by students increased both from the pre-2020 baseline to 2021 and from 2021 to 2022. While students were less likely to correctly solve math challenges on the first attempt, the average number of standards mastered increased due to higher student engagement. In short, the game format in the context of Indigenous history and agriculture appears to have served to encourage students to keep trying. Three factors related to student engagement were identified; software design; integration with culture and individualized instruction for students with special needs. Students completed significantly more math problems in games that were "open choice", that is, students could select a topic to study, compared to those that followed a linear narrative with each level gradually increasing in mathematics difficulty. Teachers repeatedly mentioned the cultural content of the software as a factor in student engagement.

Publications

  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: De Mars, A. (2022). Using game design to teach middle school mathematics (or anything else). Presentation at Technology in Education conference. Sioux Falls, SD.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: De Mars, A. (2022). Using Growing Math in the Elementary and Middle School. Presentation at TIE Conference. Sioux Falls, SD.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Hanson, C. & Sanchez, D. (2022). Growing Math: Classroom-Ready Resources to Teach Math, Agricultural Science and Indigenous Culture for Grades 3-8. Presentation at NMABE Conference. Albuquerque, NM.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Hanson, C. (2022). Growing Math Supports Native Language and Culture with Tech. Presentation at Indigenous Education Research Conference. Virtual Presentation.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Burns Ortiz, M. & Taken Alive, J. (2022). Integrating Indigenous Culture Across Curriculum with Growing Math. North Dakota Indian Education Summit. Bismarck, ND.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Burns Ortiz, M. & Sanchez, D. (2022). Growing Math: Classroom-Ready Resources to Teach Math, Agricultural Science and Indigenous Culture for Grades 3-8. National Johnson-OMalley Association Conference. Las Vegas, NV.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: De Mars, A. (2022). Entrepreneurship, Native American Communities and Game Development for Scientists and Engineers. Presentation at American Indians in Science and Engineering Society conference, Palm Springs, CA.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Burns Ortiz, M. & Sanchez, D. (2022). Engaging Students Around Indigenous Language and Culture through Video Games. National Indian Education Association (NIEA) Convention & Trade Show. October 7, 2022. Oklahoma City, OK.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: De Mars, A., Taken Alive, J. & Burns Ortiz, M. (2022). What We Really Need: Educators in Indigenous Communities on Raising STEM Achievement. Presentation at the International Society for Technology in Education. New Orleans, LA.
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: De Mars, A., Taken Alive, J., Ortiz, M., Ma, Z., & Wang, M. (2022). Educators Perspectives on Factors Impacting STEM Achievement in Rural Indigenous Student- Serving Schools. The Rural Educator, 43(1), 24-36.
  • Type: Conference Papers and Presentations Status: Other Year Published: 2022 Citation: Sanchez, D. (2022). Integrating Lakota and Dakota Culture Across Curriculum with Growing Math. South Dakota Indian Education Summit (SDIES). Rapid City, SD.


Progress 11/15/20 to 11/14/21

Outputs
Target Audience:Two types of audiences benefited directly from our efforts: 975 educators and 31,158 students. EDUCATORS During the project period, training was received by 975 educators. The overwhelming majority of participants were classroom teachers, with less than 25% of the participants comprised of paraprofessionals, resource specialists, administrators, after-school program staff and student-teachers. The goal of the Growing Math project was to provide math and agricultural science integrated with Indigenous history and culture and 94% of participants in training were from six targeted states: North Dakota, South Dakota, Minnesota, Oregon, Arizona and New Mexico. An additional 3% of participants, while not residing or teaching in the target states, served students from reservations in those states, including teachers in the small portion of the Navajo Nation that extends into Utah and teachers in Nebraska who taught students from the Pine Ridge and Rosebud reservations, which extend from South Dakota to the Nebraska border. The remaining 3% of participants attending training were educators in programs serving predominantly Indigenous students with an interest in replicating the project in their region. The focus was on grades three through eight and 98% of participants served students in those grades. Although the training was available to any educator in the six target states, the majority of educators in attendance taught at schools on or adjacent to reservations. The modal teacher taught in a small school with multiple grades in his or her classroom. STUDENTS During the project period, 23,399 students used resources developed by the Growing Math project. These are students who accessed the online resources directly, tracked by username. Certainly, additional students were exposed to project resources indirectly through lesson plans downloaded and used by their teachers. While schools prohibit collection of individually identifying data on students, users are connected with classroom teacher for data reports and thus aggregate data can be reported. The students served were overwhelmingly in grades three through eight, as noted above, in the six targeted states and disproportionately attended schools that were on reservations and majority Indigenous students. Only one teacher reported teaching at a school where students did not have devices to access the Internet (Chromebook or iPad) provided to students, in a remote community that did not have Internet access. Students disproportionately (over 90%) attended Title I schools, where at least 40% of the students are from low-income families. Students had Internet access at all but one school, and access was provided out of school for remote learning, hybrid learning and homework through either subsidized home Internet access, MiFI (personal hotspot devices provided by schools), cards loaded with credit for data download or buses with hotspots that parked in local communities for scheduled hours each day. Changes/Problems:Changes that occurred were not as much in scope but in order of priorities and format. A significant change in priorities occurred as a result of input from the first two cohorts of teachers. Prior to remote learning, data on student engagement and performance had not been a major issue. As one teacher had said, "I know what my students are doing, they're right in front of me." With students no longer "in front of them", teachers were far more interested in data, particularly in time on task. Thus, new reports were created that showed not only the number of problems answered correctly (standards achieved) but also the number attempted and the number of sessions that students had interacted with the platform. This was positive in that it allowed more and earlier validation of data to be used to assess impact. It did, however, cause a small backlog of lessons and educational resources to be added to the platform as our staff created and validated reports. Addition of an asynchronous training option was not planned, but as many of the sessions are recorded for our own staff training purposes, for review for quality improvement and uploaded to the web for teachers who attended the training to review, much of the resources for asynchronous training were available. With high absenteeism and some schools moving back and forth among on-site, remote and hybrid learning, teachers were time-limited even more than usual. The possibility of asynchronous training was suggested by several teachers in cohorts to date, and thus will be added as an option for the next fiscal year. What opportunities for training and professional development has the project provided?Thirty-nine teacher training workshops were offered - 37 of these online - in which teachers were trained in use of Growing Math for in-class, remote or hybrid learning. These are in addition to the 11 conference presentations reported. How have the results been disseminated to communities of interest?In addition to one peer-reviewed article, 11 conference presentations and 39 teacher training workshops, project staff have engaged in community dissemination activities, including: Presentation at Twin Cities Start-Up Week Burns Ortiz, M., Young, S. & Wulf, M. "Too Many Children Left Behind: Improving Equity in Education through Innovation." . Sept. 23, 2021. Minneapolis. Exhibitors at Minot State University Pow-wow, Minot, ND April, 2021. Yuba City Pow-wow, Yuba City, CA , June, 2021 North Dakota Indian Education Summit, Bismarck, ND, July, 2021 United Tribes Technical College pow-wow, Bismarck, ND, September 2021 Silver City Red Paint Pow-wow, September, 2021 Cahokia Art Gallery (Native-owned art collective) , Phoenix, AZ, October, 2021 What do you plan to do during the next reporting period to accomplish the goals?As noted above, with the first priorities completed,the three issues now most requested are; more resources for middle school, more resources for special education and pre-packaging lessons into units. We are currently working on all three of these areas. We have five new educational games under development, all targeting middle school mathematics standards integrated with agricultural science, with an emphasis on traditional Indigenous aquaculture, diet, food handling, food preparation and farming. Supplemental funding to support these activities was received from four Indigenous nations. This funding will allow us to exceed the objective for number of new educational modules developed. To make the training accessible to more teachers around their schedules, we are developing asynchronous training, with short videos followed by forms for teacher assessment and input. We will be experimenting with this format for the next two quarters and offer both asynchronous and synchronous training, evaluating outcomes in terms of both teacher participation in the training and completion of educational modules by their students. With regard to impact evaluation, we will be merging teacher and student records to assess percentage of correct responses to in-app mathematics and science challenges by grade level across Fall, 2021, Spring 2022 and Fall 2022 semesters. A small amount of data is also expected to be collected in schools that offer summer sessions.

Impacts
What was accomplished under these goals? To meet these goals, the project addresses the following seven objectives: Develop a web-based portal for grades 3- 8,centered around 106 existing modules aligned with math and social studies integrated with lesson plans for food and agricultural science, where resources can be accessed directly or downloaded for lateroff-line use. This portal has been created and is hosted at growingmath.org . Each educational module includes a game, cultural component, mathematics component, agricultural science component and data collection. Modules are combined by grade level in game format of 10-20 modules each. To supplement these resources the site currently contains 82 lesson plans, 127 videos and 10 miscellaneous resources (e.g., random sample simulator). More resources are added weekly. To date, 975 educators have attended training on the Growing Math platform. Lesson plans are accessed by an average of 70 teachers per day. Educational games accessible through the portal have been accessed by over 31,000 students in the past year. 2. Conduct a multi-phase rollout of curriculum, beginning December 2020, from teacher training through implementation and formative assessment with 240 schools served by the end of the 2022 academic year. Teachers from 141 schools attended training during the first year. Rollout of the curriculum began in January, 2021. For the first six months, individual interviews were conducted with teachers following the training to gather feedback on usability and improvements. Initial interviews included "think aloud" activities of teachers using the website, games and lesson plans while verbalizing their thoughts and experiences. Based on responses from the first two cohorts (N=65), in the 2020-21 academic year, a feedback form was developed and subsequent teachers had the option of submitting their feedback in writing or through interviews with the Principal Investigator or Project Director. In compliance with social distancing protocols, over 98% of interviews were conducted via web meeting. It was anticipated at the start of the project that the highest demand from teachers would be for more curriculum, more lesson plans for at home learning. Although these were one aspect of the program teachers accessed, they placed an early priority on more data on student time on task and performance, since during the initial months of the project most students were learning from home. Thus, expanded reporting was moved ahead in the development schedule. As students have returned to school, and report features have been completed, the three issues most requested are; more resources for middle school, more resources for special education and pre-packaging lessons into units. 3. Provide technical support to schools with a maximum 24-hour response time. Frequently asked questions are answered both during the online training and on the Growing Math website. Technical support can be requested using the contact form on the project website or via email to the email address provided during the Growing Math training. Both contact forms and email address are forwarded to four staff members with overlapping hours, such that the correspondence is monitored from 7am to 9pm Central time. All questions receive a response within one business day. In over 90% of the issues, one of the staff members on the response list can answer within the current business day. For more technical issues, for example, pop-up keyboard covering the question on small, mobile devices, the issue will be referred to a developer for update, and the teacher kept informed of progress. 4. Significantly increase student completion of mathematics and science assignments, both in-class and remote learning. Prior to USDA Growing Math funding, an average of 7,662 students per year completed educational modules - a total of 38,312 students over the previous five years. In the current year alone this has increased to 23,399 students - a 200% increase over the previous year. While the number of educational modules completed is triple the previous year, the engagement of students has not declined. The average student completed 5.3 modules in prior years and 5.3 in the current fiscal year. The number of educational modules completed by students prior to USDA funding averaged 40,634 per year. In the current fiscal year, students have completed 123,743 modules, again a 200% increase over the prior year. The stable level of engagement is noteworthy as teachers reported both in needs assessment conducted prior to funding and during the current fiscal year, that overall student task completion of school assignments and homework had dropped dramatically during remote learning in the target schools. 5. Significantly increase achievement on mathematics and science standards. Year one will provide the baseline for mathematics and science standards, with year two used to assess increase. Baseline data have been collected for fall semester, 2021 and are being collected for spring, 2022. However, it must be noted that there is likely to be a history effect, in that the 2021-22 school year has been impacted by students' remote learning the prior year, multiple school closures due to COVID-19 and high degree of absenteeism, particularly due to the omicron variant. Thus, anticipated improvement in achievement in the Fall, 2022 semester cannot be attributed wholly to the impact of the project. 6. Create 24 new modules in response to formative evaluation data. To date, 20 new educational modules have been created within three educational games, Making Camp Navajo (integrating ratio and proportion with traditional Navajo farming and crafts), Making Camp Dakota (integrating division word problems with traditional Dakota diet and medicinal plants) and AzTech Empiric Empire (teaching basic statistics, fractions and basic epidemiology in the context of Latino history). An additional 21 modules are under development. Exceeding the target was enabled byfinancialsupport from four of the Indigenous nations provided training under the project. 7. Conduct multi-state analysis of impact on student performance and engagement. This objective is scheduled for year two. With nearly all data collection issues addressed, teachers trained, modules completed and platform accessible, we are well-poised to begin the summative evaluation on schedule.

Publications

  • Type: Journal Articles Status: Awaiting Publication Year Published: 2022 Citation: De Mars, A., Taken Alive, J. R. , Burns Ortiz, M. , Ma, Z. & Wang, M. (in press). Educators Perspectives on Factors Impacting STEM Achievement in Rural Indigenous Student- Serving Schools , Rural Educator.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: De Mars, A. , Taken Alive, J. R., Hanson, C. & Sanchez, D. (2021). Building STEM capacity in indigenous nations: Research, Practice and Future Plans. Presentation at the Indigenous Education Research Conference (virtual)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Burns Ortiz, M., Taken Alive, J. , Hanson, C. & De Mars, A. (2021); Growing Math: Engaging Students through integrating Indigenous culture, agriculture and mathematics. Presentation at the American Association for Teaching and Curriculum annual conference in Albuquerque, NM
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Taken Alive, J. , De Mars, A., Ma, Z. & Wang, M. (2021). Educators Perspectives on Barriers to STEM Achievement in Title I Schools Serving Rural Native American Students. Paper presented at National Indian Education Association annual conference in Omaha, NE.
  • Type: Conference Papers and Presentations Status: Other Year Published: 2021 Citation: Hanson, C. & De Mars, A. (2021). Learning math through Indigenous history and culture: The Growing Math Project. Presentation for the Din� Bi Olta School Board Association. (Virtual).
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Burns Ortiz, M. & Hanson, C. (2021). Growing Math: No One Benefits from Educational Resources They Dont Use. Din� Bi Olta School Board Association. (Virtual)
  • Type: Conference Papers and Presentations Status: Other Year Published: 2021 Citation: Burns Ortiz, M., De Mars , A.,Mondaca, D. & Sanchez, D. (2021). Everybody Has a Backstory. Noche de Ciencias, sponsored by Arlington Public Schools and USPTO. (Virtual)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Burns Ortiz, M. (2021). Addressing Barriers to Connectivity in Rural Communities, hosted by the U.S. Department of Agriculture. U.S. Department of Education ED Games Expo. (Virtual)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Gutierrez, C. , Doshi, R. & Burns Ortiz, M. Federal Opportunities to Fuel Small Business DevelopmentFostering Access, Growth, and Engagement. Presentation at Reservation Economic Summit. July 21, 2021. Las Vegas.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Taken Alive, J., De Mars, A. , Burns Ortiz, M. & Hanson, C. (2021). Building STEM capacity in Indigenous Nations: Research, Practice and Future Plans. Presentation at the National Indian Education Association Annual Convention, Omaha, NE
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Taken Alive, J. R. & De Mars, A. (2021). Resources to promote STEM achievement in schools serving indigenous students. Presented at Technology and Innovation in Education conference. (Virtual).
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Burns Ortiz, M. (2021).Games That Make You Smarter. Invited Speaker by U.S. State Department, Consulate, Istanbul. Indieway Conference. Oct. 8, 2021. Virtual.
  • Type: Websites Status: Published Year Published: 2021 Citation: De Mars, A. M., Taken Alive, J., Hanson, C. , Sanchez, D. & Burns Ortiz, M. (2020). Growing Math. https://www.growingmath.org/
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Carpenter, K., Bell, D. , Ochoa-Hendrix, J. & Burn Ortiz, M. (2021). SBIR Women Entrepreneurs Got Game, hosted by the U.S. Small Business Administration. U.S. Department of Education ED Games Expo. June 4, 2021. Virtual.