Source: The Julia Group submitted to
AZTECH: EDUCATIONAL SOFTWARE FOR RURAL SCHOOLS SERVING ENGLISH LANGUAGE LEARNERS
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
TERMINATED
Funding Source
Reporting Frequency
Annual
Accession No.
1006197
Grant No.
2015-33610-23462
Project No.
CALK-2015-00589
Proposal No.
2015-00589
Multistate No.
(N/A)
Program Code
8.6
Project Start Date
Jun 1, 2015
Project End Date
Jan 31, 2016
Grant Year
2015
Project Director
De Mars, A.
Recipient Organization
The Julia Group
2111 7th St Number 8
Santa Monica,CA 90405
Performing Department
(N/A)
Non Technical Summary
Aztech Games addresses a key national priority, "Development of technologies and services that specifically address the needs of youth and the low-income sector of the rural population" through software designed to raise the mathematics achievement of English language learners in rural communities. As this subgroup continues to rise as a proportion of the population, addressing the educational needs of rural ELL students will be an increasingly important factor in rural communities, both economically and socially.We use players' responses in a computer game format to test current mathematics performance and, within the context of a virtual world, serve up problems at the appropriate level of difficulty. Players who answer incorrectly are required to select an educational resource to study to get back in the game. The computerized format enables collection of every aspect of player instruction, from how many minutes they played the game, type of educational resource chosen and improvements in mathematics knowledge. In addition to the in-game items, a measure aligned with Common Core mathematics standards is being developed to use as a pre- and post-test to document effectiveness of the game in increasing mathematics performance.Our target market is rural schools with ELL students, whose mathematics achievement is among the lowest of all racial and ethnic groups. Raising the performance of these students has great promise for changing the trajectory of their lives, from school failure leading to high school drop out, to academic success leading to high school graduation and post-secondary education. Raising students' achievement is not only of great personal benefit to the students but the nation as a whole, given the decreases in poverty, crime and health problems associated with greater education.
Animal Health Component
0%
Research Effort Categories
Basic
10%
Applied
20%
Developmental
70%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
8066010302070%
8036010302030%
Goals / Objectives
Our goal is to develop and test an educational program, Aztech, that integrates mathematics with the teaching of ELL students in the fourth, fifth and sixth grade levels. Phase I is focused on prototype development, usability testing, creation of customized assessments and teacher resources, with baseline data collected at the end of the 8-month period.Objectives are:1. Create five playable game levels2. Create 15 in-game mathematical challenges3. Create 45 in-game instructional resources4. Create 15 on-line quizzes5. Write program to track student progress6. Conduct usability testing in two rural schools7. Analyze usability and performance data
Project Methods
EFFORTSAztech is centered on a Web-based video game interface where a bilingual avatar guides the student through collection of assessment data and completion of increasingly difficult problems in computation, data analysis and statistics concepts. The program analyzes incorrect answers and routes the student to appropriate instructional content based on the type of error identified. Instructional methods offer options of online games, quizzes, videos, animation and virtual manipulatives. Teachers can opt in to receive daily, weekly or monthly emailed reports on individual student and class performance, with links to recommended online examples, class handouts and PowerPoint presentations.In Phase I, we will focus on game and educational design with the emphasis on product creation and refinement based on usability testing feedback. Phase I data collection will focus on diagnostic, process and initial assessment data. Phase I testing will take place in two schools, one in rural Missouri with a relatively new influx of English language learners and a second in a rural California county, with a traditionally high proportion of ELL students. These schools represent the two, diverse settings in which the commercial version of Aztech will be implemented.EVALUATIONDescriptive statistics will be computed to monitor usage for the overall site and individual resources. Data collected during Phase I will be used to determine reliability of measures, improve product development, and to evaluate the determinants of student performance. Internal consistency reliability coefficients will be computed for the pretest. All quantitative analyses will be done with the SAS statistical package. The small sample size in Phase I requires that any statistical findings will be preliminary and descriptive. However, this environment holds two advantages. First, testing is occurring in the exact type of small rural school where the product is intended to be used and that is often overlooked in educational research. Second, use of a small school allows for monitoring and control of all aspects of implementation. For example, there is no concern of dilution of effects of students transferring classrooms, of inconsistency from different teachers working with students throughout the day. The teacher sample will come from six teachers from schools in rural Missouri, rural California counties and North Dakota. A purposive sample was used for maximum variation. These teachers will pilot the online orientation program. They will also participate in two online focus group sessions held over Google hangout. The first session, teachers will provide input on design of orientation and resource materials. The second session will occur after participation of all teachers in the orientation, including playing the game and review of proposed reports, and use of the game in the classroom by two teachers. The emphasis on this session will be revision of materials and design based on teacher experience.

Progress 06/01/15 to 01/31/16

Outputs
Target Audience:The target audience served by this research during the Phase I grant period is the population of rural schools serving English language learners in grades 4 through 6. The audience reached included both teachers and students who were English language learners and non-ELL students enrolled in the same classrooms. Changes/Problems:There were no major changes. A minor change was to task 2.2 "Create 15 online quizzes"The program for administering and scoring quizzes is completed, nine quizzes were completed and tested. Analysis of data from a prior study of educational game data identified perseverance in game play as a major issue (De Mars, 2015). Rather than continue with a single pattern of "instruction- quiz -return to math challenge", we experimented with varying this pattern to test the hypothesis that less predictability will increase engagement. Preliminary data support this hypothesis. What opportunities for training and professional development has the project provided?A post-conference tutorial on factor analysis using examples from data collected in this projectwas presented at the SAS Global Forum in Las Vegas with 16 attendees from universities and colleges teaching statistics. A two-hour online orientation was presented for six teachers from three states.Two undergraduate interns work on the project. How have the results been disseminated to communities of interest?Results were presented at three conferences: California Association of Bilingual Education, North Dakota STEM conference and Computer Users in Education.A bi-weekly mailing list is distributed to 458 registered members, discussing game development and resources available. Two journalarticles have been invited for publication in a special issue on serious games. Discussion of game progress is also published on the 7 Generation Games blog, which received 58, 560 visits during the grant period. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? A five-level prototype was tested by 35 students and 6 teachers in three states, providing the basis for usability testing and initial data collection. Assessment measures were developed and validated. Supplemental resources in both English and Spanish were created to support teachers and inform parents. Through usability testing we also were able to identify technical, institutional and education barriers to implementation early on and adapt to work toward solutions to overcome these obstacles. After meeting all technical Phase I technical objectives, The Julia Group has established the necessary foundation to bring to commercialization a product that addresses the performance gap for ELL students and takes into consideration the technical obstacles regularly encountered by rural schools.

Publications

  • Type: Journal Articles Status: Accepted Year Published: 2017 Citation: Burns Ortiz, M. & Davis, W. (accepted) Learning mathematic and Ojibwe culture through video games
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: De Mars, A. (2016). Impact and influence of educators in game design. Presentation at the annual meeting of the Computer Users in Education, Palm Springs, CA.
  • Type: Conference Papers and Presentations Status: Other Year Published: 2016 Citation: Buendia, J. & Burns Ortiz, M. Educational games for EL students: State of the art. Presentation at the annual conference of the California Association of Bilingual Education, San Francisco, CA.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2015 Citation: De Mars, A. (2015), Yes, PROC FREQ does that! Presentation at the Western Users of SAS Software annual meeting, San Diego, CA.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2016 Citation: De Mars, A. & Bargeron, D. (2016). Impact and influence of educators in game design. Presentation at the North Dakota STEM conference, Grand Forks, ND.