Progress 07/01/19 to 02/29/20

Outputs
Target Audience:Our target remains teachers, district leaders, principals, and, of course, students. BlocksCAD is a 3D modeling tool that fills a critical need for developing students' necessary skills that build the careers and technology of the future. Students create their own designs using block-based coding that can either be fabricated on any 3D printer or converted to an AR/VR platform. In classrooms and afterschool programs around the country, we have shown that students who engage in activities using BlocksCAD have improved attitudes towards math, engineering, and computational thinking. Some students, when presented with an alternative instructional approach through BlocksCAD have also indicated a deeper understanding of geometry, algebra and computational impact. In turn, students who learn through BlocksCAD are better prepared to meet real world challenges. Additionally, we provide extensive real-time and video-based professional development for interested educators and schools, so they can build their own skills, provide quality instruction to students, and use BlocksCAD to bolster their teaching. Unlike other products that require expensive enterprise subscriptions for schools, any individual user can use BlocksCAD for free. BlocksCAD provides a number of free sample lessons plans, covering grades 3 - 12 and BlocksCAD help videos are available on our YouTube channel, so students can access help remotely. This allows schools with less funding and lowerincome individuals on their own the opportunity to use BlocksCAD for free on any device (including a phone or tablet). In our summer camps and after school classes, students of all abilities, socioeconomic backgrounds, and comfort levels with programming and math come together to learn in various ways. With a diverse group of teachers and teaching assistants, students get small class sizes for personalized instruction. With BlocksCAD available nationwide and globally, students who may never have access to a BlocksCAD classroom, 3D printer, or coding class now have the opportunity to build coding and math skills. Changes/Problems:Unfortunatley were were not able to complete all the assesment we had hoped to accomplish due to the unexpextected early school closures in Kentucky due to the Corona Virus pandemic. What opportunities for training and professional development has the project provided?During the course of our work we trained 10+ Middle and High School teachers on using BlocksCAD to enhance their math and computer science teaching How have the results been disseminated to communities of interest?The work we did with the pilot schools in Kentucky was shared on social media as well as our website. Some of the projects students made were highlighted in our BlocksCAD gallery What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? (1) Build a data model to support collecting fine-grained student interaction data and connect the BlocksCAD tool to SCALE. The data model and event system needed to store student model history within BlocksCAD has been completed. The event system tracks user interface events (clicks, drags, typing), decomposes them into event types (creation/deletion/editing of code, moving of blocks, reconnecting of blocks, undo/redo events, model rendering, other user interface events), and stores a history of the projects that can sent in real time to SCALE, with the flexibility to be used in the future to 'replay' how a model was created or allow users to 'reset' their projects to an earlier point. This work used actual data from use of BlocksCAD software in the classroom. (2) Develop the suite of teacher management tools and functionality for a prototype BUILDERS Classroom and teacher portal. BlocksCAD has designed, implemented, tested, and rolled out to production 'BUILDERS Classroom', our teacher portal for classroom management within BlocksCAD. Our database architecture and backend API was expanded to include the concepts of classrooms and assignable lessons, and both project and user database records were expanded to support classroom management, assignments, and project history tracking, along with the necessary API code to easily enable use and management of this data. Teachers can now create and manage 'classrooms' in BlocksCAD that lets them see groups of students, manage their BlocksCAD accounts, and access student projects. Student rostering has been implemented to allow both manual student import, allowing students to join via an invitation, and automatic rostering through integrations with Google Classroom. The BUILDERS Classroom interface includes a new teacher web portal. The top level allows teachers to manage classrooms - create, edit, load students, and delete them. Each classroom from the top level page leads to a tabbed page for that classroom (the 'class detail' page). From here, teachers can manage student accounts from the "current snapshot" page, manage student projects from the "download work" page, assign lessons to the class from the "Lessons" page. (3) Develop curricular materials to be used in the in-class pilot tests. Three activities were designed to provide students an initial experience in using BlocksCAD: Robot, Cube Cage, and Snowflake Generator. These projects were designed to be scaffolded for ease of student use, but also to allow teachers who had no coding, engineering, or CAD experience to be able to assign and follow along with the activities. Each of these projects are aligned with common core state standards for math. In particular, they all are examples of using shapes and translations. Additionally, depending on how the teacher uses the lesson plans, for example: Standard 6.EE.A.1 -- Write and evaluate numerical expressions involving whole-number exponents. Students can use exponent blocks to calculate when constructing projects. Students can use exponents to define the dimensions of their shapes, which visually demonstrates how each power a dimension is raised by will result in exponential growth in the size of their shapes. Shapes can be 3D printed to have their volume calculated, or measured through liquid displacement. (4) Conduct in-classroom tests of the BUILDERS Classroom prototype and prototype curricular materials, and collect interaction data on a set of exemplar student assignments. In order to test the prototype in the classroom we first developed a teacher survey and created a detailed interview instrument to allow us to better understand how the BUILDERS Classroom would integrate with current classroom practices. Our data collection and analysis included pre and post test data with usage of the BlocksCAD tool. We utilized the BROMP protocol for collecting classroom observations, and performed over 400 observations in the 4th and 6th grade classrooms using the BlocksCAD tool. Overwhelmingly, students' most common affective state was concentration (66%) and the least common observed state was boredom (7%). Across both grades, students remained focused on the task a majority of the time, with an observed 83% of on task behavior. These results show positive engagement effects in using the system compared to traditional classroom environments. Our post-assignment focus groups with participating teachers were delayed and then canceled with the closing of the schools based on the COVID-19 pandemic. We were able to survey teachers and while the response rate (n=4) is not valid for quantitative analysis, the teachers provided a wealth of qualitative insights in their open feedback. This included ways to further help learners, such as "providing more details for younger learners on the worksheets". They also provided feedback that demonstrated the success and support of BlocksCAD, indicating that they received "fast feedback when I would ask questions online and needed support" and how the students continued using the system on their own, "I know kiddos did some for fun on their own". The surveyed instructors also provided features we could implement in the future to enhance the system, such as "I would like to see motion added into the coding". (5) Analyze Hint Factory methods and hint coverage for data collected in objective (4). The deliverables for this technical objective included loading data and creating student paths for use in the implementation of the Hint Factory method. The foundation of the Hint Factory framework consists of the Markov decision process (MDP) generator and the hint provider. All student steps for an assignment are put into a graph, such that each step is a separate state in the graph, and there can be multiple pathways towards a correct solution. This graph will capture all the different ways that students can correctly solve a problem or assignment. Using data collected from 130 students working on the "snowman" problem in BlocksCAD, we were able to build the initial state graphs. The graphs contained 852 unique states, generated from over 26,000 transactions. Of the students who adequately solved the problem (112 out of 130), there was hint coverage 73% of the time. This mirrors previous work (Fossati et al., 2009) and (Barnes et al., 2008). This result leads us to believe the implementation is a viable option for the Phase II design. (6) Refine and further develop curricular material that leverages successful student projects and assignments, and feedback from objective (4). This task encompasses iterating improvements for the curricular material, based on feedback and evaluation of the initial curriculum elements from task (4). Therefore, additional evaluation (in-class and survey) for this stage will occur in Phase II of this project. Based on feedback from teachers and students gathered during objection (4), as well as our evaluation of student engagement, and in addition to our ongoing process of refining, improving, and expanding the BlocksCAD system and associated curricular materials, we have added numerous features and lesson plans during this Phase I project. These improvements include not only expanded offerings, but also numerous guides, reference sheets, and more robust instruction on how to get the most impact out of BlocksCAD in the classroom. Iterative improvements have also been made in the UI/UX (User Interface / User Experience) aspects for BUILDERS Classroom. The editor toolbar now has a new experimental feature, for triangle blocks. Intercom messaging system was added to the editor for support - in line with this messaging system are automatic messages added to key pages, to suggest support to new teachers and encourage communication.

Publications

Progress 07/01/19 to 02/29/20

Outputs
Target Audience: Our target remains teachers, district leaders, principals, and, of course, students. BlocksCAD is a 3D modeling tool that fills a critical need for developing students' necessary skills that build the careers and technology of the future. Students create their own designs using block-based coding that can either be fabricated on any 3D printer or converted to an AR/VR platform. In classrooms and afterschool programs around the country, we have shown that students who engage in activities using BlocksCAD have improved attitudes towards math, engineering, and computational thinking. Some students, when presented with an alternative instructional approach through BlocksCAD have also indicated a deeper understanding of geometry, algebra and computational impact. In turn, students who learn through BlocksCAD are better prepared to meet real world challenges. Additionally, we provide extensive real-time and video-based professional development for interested educators and schools, so they can build their own skills, provide quality instruction to students, and use BlocksCAD to bolster their teaching. Unlike other products that require expensive enterprise subscriptions for schools, any individual user can use BlocksCAD for free. BlocksCAD provides a number of free sample lessons plans, covering grades 3 - 12 and BlocksCAD help videos are available on our YouTube channel, so students can access help remotely. This allows schools with less funding and lower-income individuals on their own the opportunity to use BlocksCAD for free on any device (including a phone or tablet). In our summer camps and after school classes, students of all abilities, socioeconomic backgrounds, and comfort levels with programming and math come together to learn in various ways. With a diverse group of teachers and teaching assistants, students get small class sizes for personalized instruction. With BlocksCAD available nationwide and globally, students who may never have access to a BlocksCAD classroom, 3D printer, or coding class now have the opportunity to build coding and math skills. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? We pride ourselves on our robust professional development and training opportunities to ensure teachers using BlocksCAD feel prepared and ready to be successful with their students. Before teachers begin implementing BlocksCAD in their classrooms, we provide two 3-hour, free courses for teachers to learn BlocksCAD basic and how students learn computer science using BlocksCAD. These are especially important for teachers who may have no formal or informal coding experience and jumping into a block-based environment may incur a great deal of stress, if they don't have any type of preliminary support. After the courses, teachers receive a 2-hour one-on-one training with a curriculum developer and instructor, so they can learn the ins and outs of the platform, work through lessons together, and ask questions before they begin using the product in their classrooms. After the training, if teachers need further support, we gladly set this up as well, so they continually feel supported throughout their use of BlocksCAD. We also have free how-to videos for 21 of our blocks, manuals available to remind teachers how each part of the editor works, and regular webinars for beginner and intermediate educators to get refreshers and learn new skills in BlocksCAD. For Hour of Code, as part of CS Education Week, we offered four webinars for educators globally to get the basics and also more advanced skills in BlocksCAD. How have the results been disseminated to communities of interest? We have submitted a proposal to present at an education innovation event and we have identified several other education and STEM education events to propose presentations and exhibit. We will also submit papers to relevant academic conferences once the data analysis is completed during the remainder of the Phase I project. What do you plan to do during the next reporting period to accomplish the goals? In the coming reporting period, we will collect more data on how our product impacts math scores in low-performing and rural schools across the United States. We recently began working with schools across Pennsylvania and Kentucky to assess how our math series and introductory builds will be able to improve scores across these schools. Additionally, with our pre and post-tests, we will be able to see what areas of the program need to be shifted to better suit students with lower visual processing skills. We are working with Mindprint, a social emotional learning organization, and Dr. David Uttal from Northwestern University's cognitive science department to create a course geared more toward students who struggle in computational thinking and spatial reasoning more profoundly, and we will be using Mindprint and other psychometric tests to recruit these students. This is an attempt to ensure our product meets the needs of all students, not just high performing students in STEM. We will be completing the data analysis as described in our proposal from quantitative data collected. We will also complete the surveys and teacher focus groups to gather and analyze qualitative data and usability feedback in order to apply a mixed method of analysis in order to gain a more thorough understanding of the educational gains for students along with any refinements we may want to incorporate into the product. We will also assess the Hint Factory coverage for assignments used in the study to develop a plan for incorporating the real-time, just-in-time hints for the assignments provided through the BlocksCAD CLASSROOM interface

Impacts
What was accomplished under these goals? We have already accomplished many of the goals listed above. Students can now access their designs from any device with wifi, including a phone or tablet. Students can use old blocks, as well as updated blocks for math-specific curricula, to create realistic models, large-scale projects, and complex designs geared toward grades 3-9 with scaffolding for younger and older grades as well. Teachers can more easily than ever implement BlocksCAD in their classrooms with our 50+ hours of CSTA and Common Core-aligned lesson plans available with their subscription, including student worksheets, sample solutions, and scaffolded activities to get them started. Teachers assign lessons to students after walking them through beginning thought and design experiments to get them thinking outside the box from the outset. We now have lessons for grades 3-8, with new Next Generation Science Standards-aligned lessons coming out weekly filling in gaps we previously had and creating more lessons for 3-5 and 6-8. In terms of data collection, we created and administered pre and post assessments to over seven different cohorts of students participating in our after school program and have begun using these assessments with our in-school cohorts as well. We conducted on-site kick-off meetings with school administrators and potential participating teachers to provide an overview of the research and development project as well as the BlocksCAD product. We then provided our preparation online training materials which was followed by hands-on remote group training sessions for the participating teachers at each school. Finally, we offered to all participating teachers and provided as requested, one-on-one mentoring sessions to ensure that teachers were well-prepared and confident to introduce the BlocksCAD curriculum to their students within the context of and in support of their regularly scheduled subject matter. Details of our professional development offerings are described in more detail in the training and professional development section of this report. Administered by teachers and analyzed by our team, we are able to see how BlocksCAD benefits students, with overall statistical significance in student understanding growth from pre to post assessment just five classes, or one session, into their use of BlocksCAD. Since beginning to use these assessments, we have worked with our curricular team to create new lessons to target standards we were missing or missing the mark on in our current curriculum. We've also begun testing updates to our editor platform to allow for deeper visualizations and larger workspace for students to render their work.

Publications