Source: UNIV OF NEW MEXICO submitted to NRP
INTERACTIVE LEARNING ASSESSMENT SYSTEM (ILAS)
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
Reporting Frequency
Annual
Accession No.
0230054
Grant No.
2012-38422-19836
Cumulative Award Amt.
$280,000.00
Proposal No.
2012-01882
Multistate No.
(N/A)
Project Start Date
Sep 1, 2012
Project End Date
Aug 31, 2015
Grant Year
2012
Program Code
[NJ]- Hispanic Serving Institutions Education Grants Program
Recipient Organization
UNIV OF NEW MEXICO
(N/A)
ALBUQUERQUE,NM 87131
Performing Department
Individual, Family, and Community Education
Non Technical Summary
The University of New Mexico will design an online Interactive Learning Assessment System (ILAS) to allow instructors to build virtual counseling cases. The ILAS provides instructors with feedback on how students learn, allowing them to improve teaching. By improving teacher instruction and students' professional identities, we aim to recruit and graduate more underrepresented nutrition students.
Animal Health Component
100%
Research Effort Categories
Basic
(N/A)
Applied
100%
Developmental
(N/A)
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
7035010101025%
7035010302010%
7245010101025%
7245010302010%
8065010101020%
8065010302010%
Goals / Objectives
As a result of strengthening educational capacities and curricula through ILAS, undergraduate students enrolled in Nutrition through Lifecycle, Management in Dietetics 1 and Management in Dietetics 2 (N~95), especially Hispanic students (20-40% of total students), will: 1. acquire deeper content understanding tied to specific contexts, such as childhood obesity and food safety, including multiple perspectives on causes and approaches to treatment/prevention 2. develop practical counseling skills in a safe environment 3. develop improved information literacy skills, specifically, locating and evaluating web-resources 4. reflect on their learning and become more metacognitive about their learning As a result of strengthening the educational capacities and curricula through ILAS, Nutrition Faculty will: 5. adapt /redesign cases to be more culturally relevant, using student voices, perspectives, and pre-test data 6. develop cases for other courses or contexts As a result of the use of ILAS in high school and science gateway courses, high school students (N~50) and freshman/sophomore undergraduate students (N~200), especially Hispanic students (~30-50% of total students) will have: 7. increased awareness and interest in the nutrition field 8. increased nutrition content understanding As a result of enhancing the educational equity for underrepresented groups through ILAS, undergraduate students enrolled in Nutrition through Lifecycle, Management in Dietetics 1 and Management in Dietetics 2 (N~95), especially Hispanic students (20-40% of total students) will: 9. develop a professional identity
Project Methods
Phase 1. Design and Development (5 mo, Sept-Jan FY13): Conceptual design (1 month): Drs. Svihla and Yakes will oversee conceptual design of cases in coordination with Hispanic nutrition students. The technology team and Dr. Svihla will begin work on the conceptual design of the ILAS. Research Planning (1 mo): Drs. Svihla and Yakes will coordinate with the evaluation team to develop/review measures and interview questions aligned to project and learning objectives. Any necessary modifications to the IRB research protocol will be made. Technology development and beta-testing (4 mo): The technical development of ILAS will involve infrastructural development of an authoring system and learning environment. The authoring system development will include creating an easy to use graphical user interface (GUI) for the development of cases, including upload of content files (images, video, text, documents), the development of branching points of assessment, and a visual representation of the branches. The learning environment development will include virtual mentors and clients designed to provide guidance and mechanisms for logging student activity and responses. Beta-testing will ensure that data are captured and exported, accounts can be password protected. Content development (4 mo): Content (videos, audio files, images, text, and documents) will be developed based on the conceptual design of the example cases. Cases will be developed using the authoring system. Initial case development will focus on cases for use in the spring course, Management in Dietetics II. Phase 2. Deployment and evaluation (6 mo, February-June FY13): Initial implementation will occur in Management in Dietetics II. The evaluation team will assess the impact, focusing on how ILAS may be effectively embedded in context. For research purposes, data collection and analysis will be undertaken. Phase 3. Evidence-based refinement (3 mo, July-September FY13-14): Based on the pilot implementation, the cases will be adapted for use in high school settings and other nutrition and science classes. Further case development will occur to support the fall courses, Nutrition through the Lifecycle and Management in Dietetics I. Phase 4. Initial dissemination (3 mo, July-Sept FY13-14): Conference publications, a project website, and demonstrations will be used to promote our design and approach. Phase 5. Redeployment and evaluation (9 mo, Oct-June FY14): Implementation of refined cases will occur in the fall and spring nutrition courses. Any partnerships established through dissemination efforts will be contacted for interviews/further data collection. Evaluation strategies discussed in the narrative and refined with the evaluation team will be used to assess the impact. Phase 6. Dissemination and Publication (5 mo, Mar-July FY14): Concurrent to Phase 5, peer-reviewed publications will be prepared and will target journals focused on learning technologies and education of nutrition/dietetics students. Updates will be posted to the project website, including example cases, instructions for creating and adapting cases, and details about the technology specifications.

Progress 09/01/12 to 08/31/15

Outputs
Target Audience:In the first year of the project (2012-2013), two Interactive Learning Assessment (ILA) cases were developed and used in the undergraduateNutrition through Lifecycle course (TotalN=42/14 Hispanic) andgraduate Nutrition Assessment course (Total N=20/4 Hispanic). In the second year of the project (2013-2014), 7 ILA cases were developed and used in the undergraduate Nutrition through the Lifecycle course (Total N=47/16 Hispanic) and 2 ILA cases were developed and used in the online Vitamins and Minerals in Human Nutrition course (Total N=50/11 Hispanic). In the third year of the project (2014-2015), 7 ILA cases were used in the online undergraduate Nutrition through the Lifecycle course (Total N=47/16 Hispanic) and 1 ILA case was used with students at the Health Leadership High School in Albuquerque (Total N=53/48 Hispanic). In addition, the ILA platform was used as a teaching tool for students enrolled in Secondary Science Teaching Methods (2012-2013; 2013-2014); Advanced Instructional Strategies (2012-2013; 2013-2014), Project-Based Learning (2012-2013), and Instructional Design (2014-2015). The total number of students/Hispanic students using the ILA platform each year was: 2012-2013 (24/13); 2013-2014 (20/8); and 2014-2015 (8/3). It was also introduced as a teaching tool to ten active teachers at the Health Leadership High School in Albuquerque. There has been interest in using the ILA platform in other universities that are a part of the Western MCH Nutrition Partners (UCLA; Colorado State University; Arizona State University; University of Washington; Oregon Health & Science University; University of California, Berkeley). In addition, the NM WIC program is interested in the platform, and New Media and Extended Learning at the University of New Mexico has expressed an interest in the project. Changes/Problems:Unfortunately, we were not able to reach the number of freshman and sophomore undergraduate students that we had planned. We had hoped to achieve this through collaboration with theUNM Academic Science Education and Research Training (ASERT) program for postdoctoral fellows,which provides three years of support to fellows wishing to achieve excellence as both educators and research scientists in biology, bioengineering and biomedical sciences. These fellows often teach freshman and sophomore undergraduate courses. Although ILA was well-received by the ASERT fellows, the technical problems that we were having with the platform precluded its use with this group before the end of the current grant. What opportunities for training and professional development has the project provided?1. Undergraduate and graduate nutrition students received additional training in nutrition content and counseling, as part of our project goals. 2. The PD has been able to work closely with the Co-PD, who is an expert in teaching and learning. In addition, the Co-PD has been able to access the nutrition content knowledge of the PD for projects such as the collaboration with the Health Leadership High School. 3. The interdisciplinary team includes undergraduate computer science students and graduate nutrition students who have had opportunities to: (a) learn about working on an interdisciplinary team; (b) present at conferences; (c) plan and conduct studies using ILA for the fulfillment of master's degree requirements; (d) conduct literature reviews of relevant research under the mentorship of the PD and Co-PD; (e) attend regularresearch group meetings. One nutrition graduate student involved with the project later completed his dietetic internship, and stated that his experience with the ILAS project has been a significant asset. He said the time he spent imagining and researching cases, thinking about how an interview might go and what issues might arise, has helped him in real-life interview scenarios during the internship. He said:"In creating these and thinking about these, the act of imagining them and trying to make them real, I had the scenario in my mind a few different ways, so when I encounter something like that at the hospital or the WIC clinic, I've at least practiced thinking about this." How have the results been disseminated to communities of interest?The PD presented ILA research to the New Mexico Special Supplemental Nutrition Program for Women, Infants and Children (NM WIC), and to theWestern MCH Nutrition Partners (UCLA; Colorado State University; Arizona State University; University of Washington; Oregon Health & Science University; University of California, Berkeley). Both NM WIC and the Western MCH Nutrition Partners are interested in using the ILA platform to develop new cases, and we will be sharing it with them. The study team presented ILA to New Media and Extended Learning (NMEL) at UNM, and they also expressed interest. Becky Adams, the Director of Faculty Services and Online Course Development, stated thatshe and others in her department who have seen demonstrations of the ILAS are very excited about it, and would like to integrate it into UNM's toolkit of online learning options. She said they have seen case-based or simulation-based software at conferences before, but they are very expensive, so it's exciting to see something homegrown to fill that need. She said several UNM colleges and departments have expressed a desire for an online tool for creating case studies and simulations, and UNM doesn't currently have anything that does that.She stated, "It is very sound, deep pedagogy...learning through a simulation or a case study or a complex problem or problem-based learning ... I think that's the thing that jazzes us as educators. It's better; you get it deeper. As a student, you're going to remember more if you've done it. It's a solid pedagogy that we really, really want to be using. And then if you have a technology or an application that allows a faculty member to do it who isn't technology savvy, now we've all won. I think that's the strength of it." The results have also been disseminated at new graduate student orientation and faculty meetings, and via student and faculty presentations at local, regional and national conferences focused on nutrition, learning and technology, and through education and nutrition peer-reviewed journals. The results are also disseminated through the project website: https://sites.google.com/site/iddealab/iddea-lab-projects/interactive-learningassessment. What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? Overall, we reached double the number of students in the UNM Nutrition Program than we originally anticipated. From 2012-2015, 206 nutrition students (61 Hispanic;30%)completed an ILA case in the multiple courses/contexts (Nutrition Through the Lifecycle, Vitamins and Minerals in Human Nutrition or Nutrition Assessment courses). 1. Students acquired deeper content understanding related to a number of topics, including preconception nutrition, gestational diabetes, complications of prematurity, breastfeeding, childhood obesity, children with special health care needs, eating disorders, and malnutrition in the elderly. All casesincludedmultiple perspectives on causes and approaches to treatment/prevention. It was confirmed using delayed post tests that students performed similarly on content learned from ILA and from lectures. One student said:"I liked that the case study gave me a huge list of things that we had, you know? I mean, as you're doing the case, you're learning these are the issues, the main problem, the fatigue, the time, the hypertension, the multiple medications" and she learned that she had to prioritize them, rather than trying to address all issues at once.Another student said:"It felt good to be able to recognize [the patient's] health issues a little more comfortably after our work here and the information given in class." 2. Students developed practical counseling skills in a safe environment and also agreed that the case resembled a real life situation; they overwhelmingly reported that they would use what they learned in other classes and in their careers, and that what they learned was important for their future professional work. Sample quote: "I was moved by all the cases. [...] I got the impression--I felt like I was a part of this person's recovery. That I was teaming up with a registered dietitian who was taking care of this client."Another samplestudent quote:student recognized thatinterventions need to be tailored to individuals, saying "actually speaking with someone and learning a little bit about what their life is like and what things they already feel capable of, what things they feel are already beyond their ability currently and figuring out, okay, knowing about this person and what they are already capable of, what are the most important things to do right now." 3. Students developed improved information literacy skills; specifically, using web-resources. 4. Students reflected on their learning and became more metacognitive about their learning. Most students felt confident that they mastered the content of the case. Sample student quote: "I liked that I had to think about my responses but then after I submitted my responses I got to see how the instructor interpreted the data (as she has more experience than I do in dietetics). There was a lot of information provided but it was a nice way to learn because I was actually interacting with the information instead of just hearing it." 9. Students developed a professional identity. Astudent explained that she enjoyed going through the "cases because this is exactly what I want to be doing right now. The only way we are going to become good practitioners is to get practice and so if we can't see real patients, this is the next best thing." We reached an additional 52 students (24 Hispanic) through the Teacher Education (pre-service high school science teachers)and Organization, Information and Learning Sciences Programs. These students have studiedthe ILA platform and examplecases as learning tools that they can use in their classrooms.They were excited about the format and believed the case study approach would help their learners place themselves in the role of the professional. They liked that it provided a "safe environment to make mistakes, get feedback and learn professional skills." We also met our goal for reaching high school students with ILA. OneILA case was used with students at the Health Leadership High School in Albuquerque (Total N=53/48 Hispanic). The Co-PD/PD worked withtwo high school teachers from Health Leadership High School, an Albuquerque not-for-profit charter school that serves students who have been marginalized by traditional schooling approaches, to adapt a case on gestational diabetes that was used for the undergraduate nutrition classes. An additional 8 teachers at the Health Leadership High School were trained on ILA and may use it for future projects. 7. Students increased their awareness and interest in the nutrition field.Students connected what they were learning in the case to the real work professional dietitians do and to their everyday lives. For instance, one student asked:"once you've had gestational diabetes, does that mean you are likelier to get it again next time you are pregnant? Because my mom is pregnant now, and she had gestational diabetes with my sister. I'm going to make sure she goes in for her check up and asks about it this time." Anotherstudent reflected that she didn't realize that when considering nutrition, she should "take the processed foods into account." 8. Students increased their nutrition content understanding. They calculated BMI and recognized that the client in the case was gaining weight too quickly. For instance, one group accurately explained, "I think she is not on track with the weight gain she has had because she has gain 1.8 lbs each week and that's not good either for her or the baby." Most of the students were initially confused about the different types of diabetes, believing that type 1 could lead to type 2 diabetes, and that ababy could get diabetes at birth from the mother having gestational diabetes. Based on their responses to the case, and follow-up feedback provided to the teachers, many of them made learning gains in this area. For instance, one group explained "the risks for Rosario and her baby are...type 2 diabetes later in life," and another group explained that there was a "risk of having diabetes type 2 when the baby gets older."

Publications

  • Type: Journal Articles Status: Published Year Published: 2014 Citation: Svihla V. Advances in Design-Based Research. Frontline Learning Research. 2014; 2(4): 35-45. Available at: http://journals.sfu.ca/flr/index.php/journal/article/view/114/212
  • Type: Conference Papers and Presentations Status: Published Year Published: 2014 Citation: McKay T, Cantarero A, Svihla V, Yakes Jimenez E, Castillo T. Becoming a Professional through Virtual Practice. Learning and becoming in practice: The International Conference of the Learning Sciences (ICLS). 2014; 3: 1563-1564. Available at: http://www.isls.org/icls/2014/downloads/ICLS%202014%20Volume%203%20(PDF)-wCover.pdf
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2015 Citation: Donahue M, Donahue J, Yakes Jimenez E, Castillo T, Svihla V. Needs-based Designing for Learning Professional Practices: Scholarly and Worldly. Paper presented at the AERA, Chicago, IL, April 19, 2015 (2:15-3:45 PM).
  • Type: Journal Articles Status: Submitted Year Published: 2015 Citation: McKay T, Yakes Jimenez E, Svihla V, Castillo Tim, Cantarero A. Teaching Professional Practice: Using Interactive Learning Assessments to Simulate the Nutrition Care Process (NCP). Submitted to Topics in Clinical Nutrition. 2015.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2013 Citation: Yakes E, Cantarero A, McKay T, Svihla V, Castillo T, Valdez I, Hertel J. Interactive Learning Assessments. Paper presented at the Fifth Annual Innovations in Practice & Education at FNCE 2013 Conference, Houston, TX.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2013 Citation: Svihla V, Yakes E, Castillo T, Cantarero A, Valdez I, Dominguez N. Interactive Learning Assessment: Providing Context and Simulating Professional Practices. Paper presented at the Games, Learning, Society 2013, Madison, WI.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2014 Citation: Svihla V, Yakes Jimenez E, Castillo T, McKay T. Interactive Learning Assessment: Simulating Professional Practice. Invited session. Elevating Science and Education: IRACDA 2014, Albuquerque, NM.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2015 Citation: Yakes Jimenez E, Svihla V, Castillo T, McKay T, Cantarero A. Interactive Learning Assessment: Simulating Professional Nutrition Practice. FASEB J. 2015; 29:909.5.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2014 Citation: Jimenez E, Svihla V, Castillo T, McKay T, Cantarero A, Baumgardner K, Kelley P, Hernandez I. Interactive Learning Assessment: Simulating Professional Practices. Research presented at the 2014 Agri-Science Education for the 21st Century Conference, Miami, FL.


Progress 09/01/12 to 08/31/13

Outputs
Target Audience: In the first year of the project, two Interactive Learning Assessment (ILA) cases were developed and used in Nutrition courses: (1) Undergraduate students enrolled in Nutrition through Lifecycle (N=42), including Hispanic students (n=14) completed an ILA case related to childhood obesity; (2) Graduate students enrolled in Nutrition Assessment (N=20), including Hispanic students (n=4) completed a case focused on anemia assessment. Changes/Problems: Originally, we intended to implement ILA cases in the Management in Dietetics I and II courses at UNM; while this will likely eventually be achieved, it may not be possible during the grant period because of the time/effort involved in developing the cases and the authoring system. We still anticipate reaching more than 100 Nutrition undergraduate students. What opportunities for training and professional development has the project provided? 1. Undergraduate and graduate nutrition students received additional training in nutrition content and counseling, as part of our project goals. 2. The PD has been able to work closely with the Co-PD, who is an expert in teaching and learning 3. The interdisciplinary team includes undergraduate computer science students and graduate nutrition students who have had opportunities to: (a) learn about working on an interdisciplinary team; (b) present at conferences; (c) plan and conduct studies using ILA for the fulfillment of degree requirements; (d) conduct literature reviews of relevant research under the mentorship of the PD and Co-PD; (e) attend bi-weekly research group meetings. 3. The PD, Co-PD, and one graduate student presented ILA research to the UNM Academic Science Education and Research Training (ASERT) program for postdoctoral fellows, which provides three years of support to fellows wishing to achieve excellence as both educators and research scientists in biology, bioengineering and biomedical sciences. 4. The Co-PD presented an ILA case to her pre-service science teachers (n=9). 5. The Co-PD presented ILA research to the New Mexico Discipline-Based Education Research Group Journal Club. 6. The PD presented ILA research to the New Mexico Special Supplemental Nutrition Program for Women, Infants and Children (NM WIC), and NM WIC plans to use ILA cases to train staff on gestational diabetes. How have the results been disseminated to communities of interest? The results have been disseminated at new graduate student orientation and faculty meetings, and via student and faculty presentations at local, regional and national conferences focused on nutrition, learning and technology. The results are also disseminated through the project website: https://sites.google.com/site/iddealab/iddea-lab-projects/interactive-learning-assessment. What do you plan to do during the next reporting period to accomplish the goals? We plan to complete beta-testing of the Interactive Learning Assessment (ILA) case authoring system, which will allow others to develop cases. The PD is using the ILA case authoring system to implement six new cases for undergraduate students enrolled in Nutrition through Lifecycle this Fall 2013. As a result of our dissemination efforts, many colleagues are eager to incorporate ILA cases in their teaching. For instance, a nutrition faculty member who teaches Energy Nutrients in Human Nutrition will incorporate cases into her online course this Fall 2013. A Hispanic graduate student is using the ILA case authoring system to implement the two cases for undergraduate students enrolled in this course, and will use evaluation of this work for her thesis. ASERT Fellows who teach introductory biochemistry courses plan to incorporate ILA cases into their teaching in Spring 2014. This will involve incorporating ILA into other courses or contexts. A study is planned in a local high school for Spring 2014 where students-- especially Hispanic students (~30-50% of total students) will have: increased awareness and interest in the nutrition field and increased nutrition content understanding.

Impacts
What was accomplished under these goals? In the first year of the project, two Interactive Learning Assessment (ILA) cases were developed and used in Nutrition courses: (1) Undergraduate students enrolled in Nutrition through Lifecycle (N=42), including Hispanic students (n=14) completed an ILA case related to childhood obesity: 1. Students acquired deeper content understanding related to childhood obesity, including multiple perspectives on causes and approaches to treatment/prevention. It was confirmed using delayed post tests that even students who got answers wrong during the ILA case were able to make use of the feedback presented automatically in the ILA case. Sample quote: “I liked that the case study reassured us of our answers and made us feel like there were no right or wrong answers. This helped me to really think about the questions instead of focusing on whether or not I would get full credit for being right.” Students who chose incorrect answers on the ILA still performed well on the delayed post test (a portion of the final exam). 2. Students developed practical counseling skills in a safe environment and also agreed that the case resembled a real life situation; they overwhelmingly reported that they would use what they learned in other classes and in their careers, and that what they learned was important for their future professional work (79% strongly agree; 21% Agree). 3. Students developed improved information literacy skills; specifically, using web-resources. 4. Students reflected on their learning and became more metacognitive about their learning. Most students felt confident that they mastered the content of the case. Sample quote: “I liked that I had to think about my responses but then after I submitted my responses I got to see how the instructor interpreted the data (as she has more experience than I do in dietetics). There was a lot of information provided but it was a nice way to learn because I was actually interacting with the information instead of just hearing it.” (2) Graduate students enrolled in Nutrition Assessment (N=20), including Hispanic students (n=4) completed a case focused on anemia assessment: 1. Students acquired deeper content understanding tied to anemia assessment 2. Students developed practical counseling skills in a safe environment 3. Students developed improved information literacy skills; specifically, using web-resources 4. Students reflected on their learning and became more metacognitive about their learning. Sample quote: “I liked having instant feedback from the "supervisor" to see if I was on track with my answers, and if I was interpreting it correctly.”

Publications

  • Type: Conference Papers and Presentations Status: Published Year Published: 2013 Citation: Yakes E, Cantarero A, McKay T, Svihla V, Castillo, T, Valdez I, Hertel J. Interactive Learning Assessment: Simulating Professional Practices. NACTA Journal. 2013; 57 (Suppl 1): 44.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2013 Citation: Svihla V, Yakes E, Castillo T, Cantarero A, Valdez I, Dominguez N. Interactive Learning Assessment: Providing Context and Simulating Professional Practices. Proceedings of Games, Learning, Society 9.0. 2013.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2013 Citation: Svihla V, Yakes E, Cantarero A, Valdez I, Castillo T (03/2013). Panel - Interactive Learning Assessment: Pilot in an Undergraduate Nutrition Class, presented at The Role of Assessment in Student Success: How do YOU Define Student Success?: 2013 New Mexico Higher Education Assessment and Retention Conference. Albuquerque, NM.