Aztech Phase II: Educational software for rural schools serving English learners

AZTECH PHASE II: EDUCATIONAL SOFTWARE FOR RURAL SCHOOLS SERVING ENGLISH LEARNERS

Sponsoring Institution

National Institute of Food and Agriculture

Project Status

TERMINATED

Funding Source

SMALL BUSINESS GRANT

Reporting Frequency

Annual

Accession No.

1010276

Grant No.

2016-33610-25841

Project No.

CALK-2016-04214

Proposal No.

2016-04214

Multistate No.

(N/A)

Program Code

8.6

Project Start Date

Sep 1, 2016

Project End Date

Aug 31, 2018

Grant Year

2016

Project Director
De Mars, A.

Recipient Organization
The Julia Group
2111 7th St Number 8
Santa Monica,CA 90405

Performing Department
(N/A)

Non Technical Summary
English language learners (ELL) are both the fastest growing and among the lowest performing student populations in rural schools. The average ELL student has spent seven years in U.S. school systems and is below grade level in reading, writing and math. High-poverty, minority children show gains in mathematics achievement during fourth through eighth grades when they attend schools focused on good teaching, successful instructional strategies, increased effort, better attendance and self-confidence in mathematics. Despite the influx of ELL students in rural communities, educational resources for these students are scarce, if they exist at all.AzTech Games is a suite of software applications with text read aloud to students, a storyline centered on historical events and in-game mathematical challenges and instruction. At the click of a button, written and spoken language switches from English to Spanish, providing bilingual supplemental instruction for schools that may have few, if any, bilingual staff. This technological solution is optimized for rural communities and usable by both ELL and non-ELL students, that teaches, tests and tracks progress in mathematics and English/language arts, accommodates to the student's native language and can be used by a regular (non-bilingual) classroom teacher.AzTech is centered on a computer game interface where a bilingual avatar guides the student through collection of assessment data and completion of increasingly difficult problems in computation, measurement and statistics concepts. Immediate feedback is provided to students, with reinforcement in the form of prizes earned for correct answers. The program analyzes incorrect answers and routes the student to appropriate instructional content based on the type of error identified. Instructional methods offer options of online games, quizzes, videos, animation and virtual manipulatives. Teachers can access analytic reports on individual student and class performance, with links to recommended online examples, class handouts and PowerPoint presentations. Utilizing software that enables us to create a cross-platform product, we address both the skills students need and account for hardware systems limitations commonly encountered when working in rural schools.After meeting all technical Phase I technical objectives, including prototype development, usability testing, creation of customized assessments and teacher resources and baseline data collection, The Julia Group has established the necessary foundation to bring to commercialization a product that addresses the performance gap for ELL students and takes into consideration the technical obstacles regularly encountered by rural schools.In Phase II, we are building upon our findings and expanding our research. Phase II will see the development of of 16 game levels in an online application that integrates instruction in English language and mathematics within a 3-D virtual world. To complement the software application, The Julia Group will create library of supplemental instructional resources and professional development resources for teachers. The end result will be a multifaceted commercial product on the marketplace. We will collect and analyze student performance and engagement data in a multi-year, multi-state study, with schools from three rural districts in three states already committed. In addition to efficacy data, we will collect and analyze data on barriers to implementation within rural schools the findings of which will not only further shape our efforts, but will also be disseminated in publications and presentations for the benefit of future technology companies in serving rural communities.

Animal Health Component

Research Effort Categories

Basic

Applied

20%

Developmental

80%

Classification

Knowledge Area (KA)	Subject of Investigation (SOI)	Field of Science (FOS)	Percent
806	6010	3020	50%
805	6050	3070	50%

Knowledge Area
805 - Community Institutions, Health, and Social Services; 806 - Youth Development;

Subject Of Investigation
6010 - Individuals; 6050 - Communities, areas, and regions;

Field Of Science
3070 - Psychology; 3020 - Education;

Keywords

english language learners

education

educational technology

rural schools

Goals / Objectives
THE GOAL OF AZTECH GAMES IS TO ANSWER THE FOLLOWING TECHNICAL QUESTIONS:? Can a computer-based instructional system for teaching mathematics to English language learners in rural schools:o Increase mathematics achievement scores?o Increase English/language arts achievement scores?o Improve students' engagement when learning mathematics?o Be applied by teachers in rural classrooms without reducing time available for teaching non-ELL students?? How can hardware and network speed requirements be minimized to allow a console-quality game to be playable over the average rural school network?? How can educational technology be re-designed to maximize student engagement?? Which aspects of a computer-based mathematics education system will present the greatest challenges for teachers in rural classrooms with both ELL and non-ELL students, and how can we address these challenges?THE FOLLOWING TECHNICAL OBJECTIVES WILL BE ADDRESSED:1. Develop a commercial release of 16 levels in an online application that integrates instruction in English language with instruction in mathematics statistics standards for Grades 2 through 8.2. Collect and analyze data on barriers to implementation experienced in rural school districts.3. Collect and analyze online data on students' usage of AzTech resources.4. Create a professional development and support website for schools, with videos on game usage and installation, supplemental materials including presentations, videos and worksheets, with integrated lesson plans.5. Conduct a two-year, multi-site analysis of impact on student performance and engagement.

Project Methods
SAMPLINGA maximum variation sample of three schools has been selected with one school each in the south/midwest, Great Plains and central California. The percentage of low-income students ranges from 60%-92%. Schools also vary in ethnicity. One school is predominantly non-Hispanic white(96%), one is predominantly Hispanic (99%) and one is predominantly Native American (99%).Within the two larger schools, for up to three grades (4-6) one student will be selected as an intervention group and one as a control group. In the third school, with only one classroom per grade, all of the students will be in the intervention group. All students in selected classrooms with signed consent forms will participate in the study. Two of the three sites were part of the Phase I research, and at both schools, participation in selected classes was 100%.DATA COLLECTIONAll data are collected electronically and stored with a unique identification number assigned by the classroom teacher. Usage data are collected at login, storing the frequency and duration of sessions. Responses to in-game challenges aligned to Common Core mathematics and English/language arts standards are also recorded automatically.During the first month of the school year, students in both intervention and control group will take an online 60-item test. As the test is in graduated level of difficulty and ends after three consecutive missed responses, only a small minority of students will take the entire test. After 16 game testing sessions - approximately 10 weeks - all students will repeat the standards-based test.ANALYSISAll analyses will be done using SAS statistical software. Descriptive statistics - frequency, mean and standard deviation - will be computed for usage data,both number and duration of sessions. Descriptive statistics for usage by school and group (intervention and control) will be computed to assess fidelity of implementation. Although there should not be any usage by the control group, this cannot be assumed as teachers may share an intervention perceived to be successful with other teachers in the same small school, despite researcher requests to the contrary. Analysis of Variance will be performed with minutes of usage as the dependent variable and school as the independent variable to test for differences in usage among the three schools. Total usage minutes will be correlated with pretest scores to determine whether a relationship exists between persistence in usage and achievement. A mixed model analysis will be conducted for the two larger schools with school as a random factor, group as the fixed factor and the standards test as the repeated factor to assess the impact of the intervention on test scores. A dependent t-test will be used to assess significance of difference from pretest to post-test at the smallest school where a control group is not feasible. A repeated measures Analysis of Variance will be performed with Group as the independent variable and test as the repeated variable, including all three schools.

Progress 09/01/16 to 08/31/18

Outputs
Target Audience:The primary target audience is rural schools with students who are English language learners. The primary target audience during the current reporting period is also predominantly socio-economically disadvantaged, as all participating schools in year 1 are Title I schools, that is, schools with a large concentration of low-income students. Secondary audiences are: researchers and policy makers providing services for English language learners, and other rural and non-rural school districts, particularly those serving students who are socioeconomically disadvantaged or limited in English proficiency.  Changes/Problems:The major problem that arose during the project period was the difficulty in matching students for pre- and post-test analysis. The app store through which some of the applications were delivered once we were out of the beta stage prohibits collecting personally identifying data from minors. This was not an issue in assessing student engagement, since the number of math challenges attempted was tracked by username. However, matching up pre- and post-test scores for individual students was not possible as the teachers did not administer the post-tests. IThe major reason given for failing to do so was the feeling that they were falling behind near the end of the year and were unwilling to give up a second class period for testing after having already used one earlier in the year for a post-test. Teachers at some of the participating schools did administer the tests each year. Thus, it was possible to compare the test scores for students entering sixth- or seventh-grade at specific schools before the software was implemented and after, which did show a rise in test scores. Combined with earlier controlled studies, these data provide further support for the efficacy of the application, but it is not an ideal design. In a new study underway with additional funding, the company has hired a consultant to collect data at the schools rather than relying on the teachers. What opportunities for training and professional development has the project provided?During the project period, project staff gave 15 presentations at regional and national conferences and had one article accepted for publication in an academic journal. A professional development website was developed and presented both on-site and online to teachers in ten states. How have the results been disseminated to communities of interest?The website with resources for instruction in mathematics has been visited over 295,000 times during the project period. Two companion video channels on YouTube, and TeacherTube, accessible by schools that block YouTube, have had a combined 196,000 views of videos. During the project period, project staff gave 15 presentations at regional and national conferences and had one article accepted for publication in an academic journal. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? IMPACT This project had three types of impacts. First, software products were developed that are optimized for use in rural and remote communities. This educational software can be played offline in areas with no or unreliable Internet. In small rural schools with students who speak English as a second language and lacking bilingual staff, teachers can use the same application for all students. Language of text, audio and video in the application, as well as instructional resources can be switched from English to Spanish with a single click. Second, the knowledge base on implementation of educational technology in rural areas has been increased, with data collected on barriers, usage and outcomes in 19 rural sites in nine states. Third, online resources for professional development and supplemental instruction were developed. The website and video channels developed have been accessed nearly half a million times during the project period. ACCOMPLISHMENTS OF PROJECT GOALS A total of 103 game were created, data have been collected and analyzed on implementation barriers, student usage and assessment. Data were collected and analyzed from 17 rural schools and two after-school programs in nine states. Comparison data were also collected from 12 urban schools. Over 40% of both urban and rural schools experienced technology barriers to software implementation, either low-bandwidth, limited devices or both. In many schools, the selection of educational software was not based on the educational value of the application but rather what would run on the low-end devices and slow internet available in the schools. Game play data were collected on 8,118 students who attempted a total of 31,518 in-game math challenges. Two types of games were developed, adventure games, where students followed a linear path in a 3D world and collector type games with a less-developed story line where players received points that could be spent on in-game purchases. Despite lower bandwidth requirements and shorter development time, analyses in both years found student perseverance to be significantly higher for the collector type games. Due to restrictions imposed by distributors and districts on collecting demographic data from minors, it was not possible to compare perseverance by age or gender. While the adventure games had fewer math challenges completed, this genre still had substantial engagement, with 40% of students completing three or more math challenges. In interviews, students provided the most positive comments regarding the offline feature of the games, as these enabled play in areas where Internet was unavailable and did not require usage of the limited data available on the typical mobile plan. A professional development website was developed and presented both on-site and online to teachers in ten states. To date, the site has received over 295,000 visits. The website offers videos in Spanish and English that teach mathematics standards for grades two through eight. Downloadable pdf and PowerPoint files are also available, in Spanish and English. These documents are not write-protected, so teachers can use as-is or modify for their own use. Lesson plans include printable cards, worksheets and activity instructions.The site also includes a support section with frequently asked questions and technical information, such as supported operating systems and step-by-step install instructions for Windows and Macintosh software. A private section of the site, available to teachers and administrators, provides links to reports on individual student data using the username assigned by the teacher. Data include number of sessions completed, math challenges attempted, in-game quizzes, pre- and post-test scores, and, for all of these answers given, whether the answer was correct or incorrect, and for quizzes and tests, the total answered correctly. A companion YouTube channel, hosting 170 videos, received 137,000 views. The project also maintains a channel on TeacherTube, accessible by schools where the firewall blocks YouTube, which had an additional 51,000 video views. Analysis of student pre- and post-test scores documented extremely low performance. At all but one school, fewer than half of students could pass the pretest of standards two to three years below their grade level. Schools were students used the software showed improvement in math scores, but still remained at a low level, improving from an average score on the mathematics standards test from 44% to 52%.

Publications

Type: Conference Papers and Presentations Status: Published Year Published: 2018 Citation: Burns Ortiz, M. (2018) ."So you want to be a game developer?" Panel session at Department of Education/IES Ed Game Expo, Jan. 8, 2018. Kennedy Center, Washington D.C. Burns Ortiz, M. (2018). 7 Generation Games. Internal presentation to USDA staff. USDA Headquarters, Jan. 9, 2018. Washington, DC. Burns Ortiz, M. (2018).Adapting 7 Generation Games for the Caribbean Market. Presentation to Trinidad and Tobago Ministry of Education. March 15, 2018. Port of Spain, Trinidad. Burns Ortiz, M. (2018). Leveraging SBIR Funding. Panel Session. National SBIR/STTR conference. Anaheim, CA. May 16, 2018. Burns Ortiz, M. (2018). C�mo iniciar un startup. Silibuz Hack-a-thon. Universidad San Ignacio de Loyola. Lima, Peru. July 7, 2018. Burns Ortiz, M. (2018). Mejores Juegos. Mejores Resultados Educativos. Mejores Futuros. Presentation. U.S. Embassy, Santiago, Chile. July 12, 2018. Burns Ortiz, M. (2018). Educational Games: Good, Bad and Ugly. Sitting Bull College, Standing Rock, Fort Yates, ND. August 9, 2018. Burns Ortiz, M. (2018). Advancing Your Career As Latinx in Tech. Panel Session. Santa Monica, CA. Oct. 24, 2018. Burns Ortiz, M. (2018). Digital Natives: Helping Tribal Youth Become Producers (Not Just Consumers) of Technology. Minnesota Indian Education Association annual conference. Hinkley, MN. Nov. 15, 2018. Burns Ortiz, M. (2018). Tradition Meets Tech: Teaching Native Youth Using Educational Video Games and through Game Design Principles. Minnesota Indian Education Association annual conference. Hinkley, MN. Nov. 15, 2018. De Mars, A. (2018). Mejores Juegos. Mejores Matem�ticas. Mejores Futuros. Presentation. U.S. Embassy, Santiago, Chile. July 12, 2018. De Mars, A. & Longie, M. (2018). Transition from School to Work on the Spirit Lake Nation. EPICS Native American Special Education Conference, Albuquerque, NM. De Mars, A. & Longie, M. (2018). Transition from School to Work on the Spirit Lake Nation. National Indian Education Association Conference, Hartford, CT
Type: Journal Articles Status: Accepted Year Published: 2018 Citation: De Mars, A. & Longie, E. (in press). The value of perseverance: Using Dakota culture to teach mathematics. Transmotion.
Type: Conference Papers and Presentations Status: Published Year Published: 2016 Citation: De Mars, A. & Gillette, B. (2016) . Customized Video Games as Effective Education for Tribal Youth. Presented at the annual conference of the National Indian Education Association, Reno, NV.

Progress 09/01/16 to 08/31/17

Outputs
Target Audience:  Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?In addition to the conference presentations listed under products, the project staff have conducted training on-site in two school districts in North Dakota. Small group demonstrations were provided for teachers and other staff from two schools in California, two schools in Missouri and three schools in North Dakota. Training was also conducted virtually, using Google Hangout, for one district in Montana. How have the results been disseminated to communities of interest?In addition to 11 conference presentations in the past year and the professional development mentioned above, the project was an exhibitor at two conferences International Society for Technology in Education, where, selected for the Startup Pavilion, ISTE waived 80% of the cost Best of Out of School Time conference What do you plan to do during the next reporting period to accomplish the goals?Plans are summarized below, by objective. Objective 1: Develop a commercial release of 16 levels in an online application that integrates instruction in English language with instruction in mathematics standards for Grades 2 through 8. An additional five levels have been designed and will be completed in year 2, thus exceeding this objective. In year 2, individuation of educational components will be added within games. A 'challenge' option will be added to the three easier games, giving two levels of difficulty. Both more bonus problems and a 'relax' mode will be added to the two more challenging games, giving three levels of academic difficulty.An additional 15 math problems will be added in Phase II, along with additional vocabulary problems, exceeding the grant objective. Objective 2: Collect and analyze data on barriers to implementation experienced in rural school districts.. No usage problems were observed with iPad versions at any of the sites. However, three of the sites did not own iPads. Two of these three sites were those who had no connectivity problems. While testing could be done on the project's devices, this left one of the schools with no viable option for the games. The district superintendent inquired whether the games might become available for Windows computers - an effort currently underway and that will be completed in the first quarter of the second grant year. Objective 3: Collect and analyze online data on students' usage of AzTech resources. Data collected from pretests (not part of the app) will be merged with game usage data to allow the project to collect demographic variables of grade, gender and age and compare usage by group. Given research on gender differences in the distribution of players for the two types of game genres used, and differences in mathematics difficulty across games, between group differences in usage are of particular interest. To evaluate the impact of professional development for staff in increasing software usage, an expansion of training activities is planned (see objective 4). Objective 4 Create a professional development and support website for schools, with videos on game usage and installation, supplemental materials including presentations, videos and worksheets, with integrated lesson plans On site presentations have been scheduled in the first quarter of year 2 for five sites in North Dakota and South Dakota, with date to be determined for additional training to be scheduled in Missouri. The Aztech Teacher Resources site includes videos and PowerPoint in both Spanish and English. A Spanish language version of the website, with resources to share with parents, is being expanded and will be completed in year two. While the content has all been created for the teacher support site, it is currently included in the 7 Generation Games website. This will be greatly expanded in year two to a separate teacher site. Objective 5: Conduct a two-year, multi-site analysis of impact on student performance and engagement. Four additional schools, in Missouri, California, North Dakota and Montana have been added for year two. An issue to be addressed in year two is the lack of persistence in completion of the evaluation measure. Because not all standards are included in the games, it is not feasible to simply substitute results of state standardized tests, even if it was possible to obtain individual results from schools. Both teachers and administrators are enthusiastic regarding the use of games as supplemental curriculum but resistant to using two class periods for testing. Two, concurrent, modifications are being undertaken to address this issue. First, meetings are being held with schools to emphasize the necessity of evaluation data, and explore possible means of obtaining these data without undue impact on instructional time. These include using the test results, provided to teachers, as one of the students' classroom test grades, removing necessity for teachers to create and grade another math test, and having the tests administered in the after-school program, with incentives - e.g., certificates, pencil sets - for the high scorers to encourage completion. A second type of modification is inclusion of the same problems from the pretest within the game itself, thus collecting assessment data as part of the game.

Impacts
What was accomplished under these goals? Accomplishments are summarized below, by objective. Objective 1: Develop a commercial release of 16 levels in an online application that integrates instruction in English language with instruction in mathematics standards for Grades 2 through 8. Nineteen levels have been completed for five games; three games of three levels and two games of five levels. Three games were created at the easier level of both game play and mathematics. Two games were created at the more difficult level of mathematics, with options for easier and harder game play. These two games also each included two 'bonus' challenges. To date, 60 in-game math problems have been created along with 17 problems for English/language arts. All math problems are available in Spanish or English, with switches in language just by clicking a button. During the fiscal year, 300 educational resources have been created. If answered incorrectly, each problem has 3-4 options for studying. Educational resources are also placed throughout the game as options for extra bonus points. Objective 2: Collect and analyze data on barriers to implementation experienced in rural school districts. Usage data have been collected at nine sites in four states; seven schools and two after school programs in Iowa, Missouri, North Dakota, South Dakota and Wyoming. In addition, to test usage for homework and home school families, testing was conducted in rural homes and community facilities in unincorporated rural areas in six states; California, Iowa, Missouri, North Dakota, Utah and Wyoming. Barriers to be addressed were of two types; administrative and technological. To insure that problems were due to design and not the specific game itself, for all games, both web-based and iOS versions were developed - exceeding the objective in the funded proposal which included only an iPad version. In schools, all games were tested on iPads as well as either Windows, Mac or Chromebook computers. Some of the home testing used Android tablets and others used iPads, depending on the device available in the household. School usage results Games streamed over the web encountered minimal administrative barriers. There was no need for installing on the device, so no computer administrator password was necessary and no permission required from school or district offices to install. None of the rural schools in this sample had a firewall that prevented accessing the games website nor prevented sending student assessment data to the servers. This is in contrast to the firewalls in some urban districts we have collected data, which are 'opt in' and sites have to be allowed through the firewall (De Mars, 2016). Internet access for games was a factor in six of the eight sites. The severity of the issue presented by internet speed varied greatly across the areas tested. In two sites, no problems were reported. In four sites, the games took an average of three minutes to load each level, but once the level loaded, no performance problems were observed. While three minutes may not sound like a long time, it is an inconvenient period for students to be sitting and waiting, especially for upper elementary and middle school students. Across 19 levels, this adds to nearly an hour of wait time over using all of the games. It should be noted that at these sites, teachers reported the load time 'not an issue, because we are used to it'. In two sites, the load time was 5-10 minutes per level. In these sites, teachers reported they probably would not use the games on the web. No usage problems were observed with iPad versions at any of the sites. Community Usage Results To assess usability in out-of-school settings, games were also tested in home and community settings. All were in unincorporated areas with the exception of one home in a town (population: 1,200) on an American Indian reservation. For the web-based version, usability issues were significantly greater in out-of-school settings. No usage problems were observed with iPad or Android versions at any of the sites. Objective 3: Collect and analyze online data on students' usage of AzTech resources. Programs were written to collect and analyze usage data using a combination of JavaScript, PHP and SQL. Data are maintained in an SQL database on a server hosted by Amazon Web Services. Once the project moved out of beta and was distributed through the Apple App Store an issue emerged as collecting data from minors violates the App Store policy. Two workarounds were used to meet the need of both the project and schools for data. Programs written currently provide the number of unique users for each game, total number of math challenges attempted, number of quizzes completed. Individual usage reports can also be requested. In year one, game usage data were analyzed at the school level. Toyama's (2015) experience with Microsoft efforts to leverage technology to improve education in impoverished schools are consistent with the results observed in year 1 - to a point. The four schools that used the software extensively took advantage of teacher training offered by the project. (See objective 4). Due to these results, in the latter part of the grant year, the project scheduled multiple professional development opportunities, both at regional conferences that have a heavy attendance of classroom teachers, paraprofessional and school-based (not district level) staff, and for individual school districts. Objective 4 Create a professional development and support website for schools, with videos on game usage and installation, supplemental materials including presentations, videos and worksheets, with integrated lesson plans A teacher resource site was created that includes a support site with information on system requirements, frequently asked questions and 31 educational resources (videos and PowerPoint). An additional 269 resources, in Spanish and English, have been created and are in the process of being added to the site. For rural teachers who serve as their own "information technology specialist ", a Computer Tune-up section has been created with basics of freeing space, updating applications and other minor maintenance for classroom computers. Presentations on integration with lesson plans and other uses of Aztech Games have been created and piloted with teachers in North Dakota, South Dakota and Wyoming. The Aztech Teacher Resources site includes videos and PowerPoint in both Spanish and English. Objective 5: Conduct a two-year, multi-site analysis of impact on student performance and engagement. The first wave of data collection has been completed, with the participation of three schools in Missouri, North Dakota and California. Data analysis is in progress. Data were available for the pretest measure for 165 students. Reliability of the pretest measure improved slightly from the Phase I results (Cronbach alpha = .83 versus .81 ). Consistent with Phase I, scores were extremely low. Although all students were in grades 6 -8 above, and standards addressed were grades five or below, the average score for students with complete data was 39% . Complete data were only available for 71% of students. Questions that were not answered were marked incorrect, based on the assumption that students did not answer questions because they did not know the answer or because they ran out of time to complete the test. There is a slight possibility that some of these missing answers may have been a result of dropped Internet connections. Within game data were collected from over 1,000 students. Analysis is in progress on student persistence in software use (as a measure of engagement) and performance on mathematics challenges.

Publications

Type: Conference Papers and Presentations Status: Published Year Published: 2016 Citation: De Mars, A. (2016). How we know an intervention works: Program evaluation by example. Annual meeting of the Western Users of SAS Software, San Francisco, CA.
Type: Conference Papers and Presentations Status: Published Year Published: 2017 Citation: De Mars, A. (2017). Native youth as producers (not consumers) of educational technology. Presentation at the Wyoming Native American Education Conference , Riverton, WY.
Type: Conference Papers and Presentations Status: Published Year Published: 2016 Citation: Burns Ortiz, M, 2016. Intellectual Property, Innovation, and Inclusive Entrepreneurship. United State of Women: Empowering Women Entrepreneurs Summit. Washington, D.C.
Type: Conference Papers and Presentations Status: Published Year Published: 2016 Citation: Burns Ortiz, M, 2016. Entrepreneurship: How to One up the Start Up: Successes in Growing Women-Owned Businesses. White House United State of Women Summit. Washington, D.C.
Type: Conference Papers and Presentations Status: Published Year Published: 2016 Citation: Burns Ortiz, M, 2016. 7 Generation Games: Top 50 Startups. Consumer Technology Association: Innovate and Celebrate. San Jose, CA
Type: Conference Papers and Presentations Status: Published Year Published: 2016 Citation: Burns Ortiz, M, 2016. 7 Generation Games: Startup Showcase. Tech Inclusion SF. San Francisco, CA.
Type: Conference Papers and Presentations Status: Published Year Published: 2016 Citation: Burns Ortiz, M, 2016. 7 Generation Games Demo. ED GAMES EXPO: Showcasing Games for Learning Developed through SBIR. Washington, D.C.
Type: Conference Papers and Presentations Status: Published Year Published: 2016 Citation: Burns Ortiz, M, 2016. "I Lost Track of Time": Using Technology to Promote Education Play and Flow. Bethel University. St. Paul, MN.
Type: Conference Papers and Presentations Status: Published Year Published: 2017 Citation: Burns Ortiz, M, 2017. Game On: Creating Better Educational Games. Native Innovation Conference. Flagstaff, AZ.
Type: Conference Papers and Presentations Status: Published Year Published: 2016 Citation: De Mars, A. (2016). Native culture + adventure games = higher math scores. Presentation at the South Dakota Indian Education Summit. Pierre, SD.