SCIENTIFIC REASONING STRATEGIES IN COLLEGE STUDENTS

• Sponsoring Institution: National Institute of Food and Agriculture
• Project Status: COMPLETE
• Funding Source: HATCH
• Reporting Frequency: Annual
• Accession No.: 1011403
• Project Start Date: Oct 20, 2016
• Project End Date: Sep 1, 2021
• Program Code: [(N/A)]- (N/A)

Recipient Organization
UNIVERSITY OF CALIFORNIA, DAVIS
410 MRAK HALL
DAVIS,CA 95616-8671

Performing Department
Human Ecology

Non Technical Summary
Scientific reasoning is a core component of science and mathematics education. Given that U.S. students have been ranked behind many other nations, as reported in large-scale international studies, the U.S. has focused on the development and implementation of a more extensive curriculum in K-12 education in STEM fields, aiming to foster students' learning of science concepts as well as advanced scientific processing skills. The vital need for students to acquire fundamental scientific processing skills has been stated in many national and state science inquiry standards. However, previous studies have suggested that K-12 and college students often experience difficulties in understanding and engaging in scientific thinking. Even adults (e.g., college students) experience difficulty in understanding and using scientific processing strategies. This project aims to investigate college students' scientific processing skills and to examine different instructional approaches that are hypothesized to facilitate the acquisition of key scientific reasoning strategies as well as domain knowledge. The findings of this project will improve our knowledge about college students' scientific reasoning skills and the effectiveness of different types of instructional approaches that would facilitate the acquisition of scientific thinking strategies. Guidelines for science education and STEM learning and teaching will be developed. This research project is consistent with the AES mission for promoting the health and well-being of children and families. The findings from this work on students' scientific reasoning have significant educational implications for designing curriculum and planning classroom and out-of-classroom activities and programs in STEM education.

Animal Health Component

40%

Research Effort Categories

Basic

40%

Applied

40%

Developmental

20%

Classification

Knowledge Area (KA)	Subject of Investigation (SOI)	Field of Science (FOS)	Percent
802	6010	3070	40%
806	6099	3020	30%
903	6020	3070	30%

Knowledge Area
806 - Youth Development; 903 - Communication, Education, and Information Delivery; 802 - Human Development and Family Well-Being;

Subject Of Investigation
6010 - Individuals; 6020 - The family and its members; 6099 - People and communities, general/other;

Field Of Science
3070 - Psychology; 3020 - Education;

Keywords

scientific thinking

science education

college students

learning and thinking

classroom teaching

after school programs

Goals / Objectives
The present research is an exploratory study into science reasoning, thinking and learning in college students. It will provide us with an understanding of the nature and learning of science thinking in young adults. It has been argued that the defining feature of scientific thinking is the set of skills involved in formulating hypotheses, designing experiments, and coordinating theory and evidence (Klahr, 2000; Koslowski, 1996; Kuhn, 1989, 2002; Zimmerman, 2007). The specific set of skills in scientific reasoning includes formulating hypotheses, isolating and controlling other variables, selecting an appropriate design or a conclusive test, generating experimental designs or conclusive tests, differentiating and coordinating theories and data, and reconciling existing beliefs with new evidence that either confirms or disconfirms those hypotheses.The present study was designed to examine how college students formulate hypotheses, design experiments, and interpret evidence, components central to scientific thinking. One purpose of the present study is to identify the difficulties that college students experience in formulating hypotheses and in designing valid experiments. Another purpose is to investigate how different instructional approaches facilitate how students learn these scientific processing skills. The key component of scientific reasoning involves the systematic manipulation of variables when testing a hypothesis using the control of variables strategy (CVS). It has been demonstrated that inquiry- based instruction is an effective approach in facilitating the acquisition and transfer of a hypothesis testing strategy and in teaching students to design unconfounded experiments (Adey & Shayer, 1990; Benford & Lawson, 2001; Chen & Klahr, 1999, 2008; Klahr, & Nigam, 2004; Kuhn et al., 2000; Lorch et al., 2010). Yet, the nature of the instruction that facilitates students learning how to transfer scientific reasoning skills is not yet clear. The present project further explores the effectiveness of two training approaches: Procedural instruction provides the learner with the "how" of using CVS, or the concrete steps involved in the process, while conceptual instruction provides the learner with the "why" of using CVS, or the rationale behind those steps. Each type of instruction will be provided after a pretest assessment of students' processing strategies, and the relative effectiveness of these instructional approaches in promoting scientific processing skills will be examined and compared in a posttest. A third purpose of the present study is to examine the relations between the use of scientific processing strategies and the learning of science content knowledge. Positive correlations between scientific reasoning abilities and measures of student gains in learning science content/concepts have been reported (e.g., Coletta & Phillips, 2005; Resnick and Gelman, 1985). In contrast to the traditional expectation that content learning in science helps students to develop scientific reasoning abilities, we hypothesize that effective use of scientific reasoning strategies in formulating hypotheses, designing valid experiments, and interpreting data would promote the acquisition of domain knowledge. We expect that those students who design more informative (i.e., unconfounded) experiments would be more likely to gain accurate knowledge/concepts about the causal variables in the domain than those who design confounded experiments. Furthermore, the present project also examines whether and how those students in the scientific reasoning training conditions, as compared to those in the no training condition, demonstrate superior acquisition of domain-specific knowledge (i.e., their understanding of the effects of the variables associated with domain knowledge/concepts).To summarize:• The first objective of this project is to examine the nature of difficulty that college students experience in understanding and using scientific reasoning strategies in formulating hypotheses, designing experiments, interpreting data and evaluating evidence, the central scientific processing strategies.• The second objective is to explore the effectiveness of different types of instructional approaches that would facilitate the acquisition of scientific thinking skills: procedural training (providing information about the step-by-step procedures of how to manipulate and control variables) and conceptual training (providing the rationales underlying the manipulation and control of variables). The main purpose is to investigate which type of training leads to the greatest increase in the processing strategies involved in formulating hypotheses and designing experiments.• The third and final objective is to investigate how the effective use of scientific inquiry processing strategies facilitates the acquisition of science content knowledge.

Project Methods
Experiment 1: Assessment of College Students' Scientific Reasoning Strategies.College students will participate in the experiment in a regular classroom that can accommodate group tests. A booklet containing scientific reasoning tasks designed to assess college students' scientific thinking skills/strategies in formulating hypotheses, designing valid experiments, evaluating evidence and interpreting data will be presented in the classroom setting. Examples of the tasks are presented in Appendix A. Data from participants' completed booklets will be coded and analyzed to assess young adults' scientific thinking strategies and to identify the difficulties they experienced in engaging in scientific thinking processes.Experiment 2: Examination of Possible Effects of Conceptual vs. Procedural Training on the Acquisition of Scientific Reasoning Strategies.The booklet consists of pretest, training, and posttest sessions. The pretest will include tasks (see examples in Appendix B) which depict either confounded or unconfounded experiments, and the participants' task is to determine whether each experiment represents a valid or invalid test and to explain why it is a good or a bad test. The pretest and posttest will be identical in format, and tasks used in pretests and posttests will be counterbalanced. The training session between the pretest and posttest will consist of confounded and unconfounded experiments as examples. After participants solve each task, each type of instruction will be provided in the booklet designed for each condition. For conceptual training, the instruction will illustrate why the control of other variables is important, explaining the rationale for why the controlled comparisons would allow them to draw valid conclusions about the target variable being investigated. For procedural training, the instruction will describe how to control the target variable and keep other variables constant, showing the procedural steps involved in manipulating the target variable and controlling other variables.Experiment 3: Investigation of Relations between Scientific Reasoning Strategies and Science Domain Knowledge.This experiment involves individual tests. Individual college students will participate in the experiment in a lab setting.Exp. 3A:The Water Displacement task (Appendix C) with all relevant props will be presented to individual students. The experiment consists of three phases: 1) pre-assessment of the understanding of each variable involved in the water displacement task; 2) designing experiments to test the causal effect of each variable (volume-size and mass-weight) on the amount of water displaced; 3) post-assessment of the understanding of each variable involved in the water displacement task. It is predicted that those who designed valid (unconfounded) experiments will be more likely to gain domain knowledge about the water displacement tasks (as evidenced by comparing students' initial and final domain knowledge), as compared to those who designed invalid (confounded) experiments.Exp. 3B:Participants will be randomly assigned to three conditions: Conceptual Training, Procedural Training, and No Training. The experiment consists of four phases: 1) Pre-assessment of water displacement domain knowledge; 2) Scientific reasoning training with another task (spring task or slope task); 3) Designing experiments to test each variable of the water displacement tasks; and 4) Post-assessment of water displacement domain knowledge. The three conditions differ in Phase 2, when participants receive either conceptual, procedural, or no instruction on how to design valid experiments with a task that is irrelevant to the water displacement task. We expect that students in the scientific reasoning training conditions will be more likely to design more valid experiments on the water displacement tasks, and therefore more likely to gain accurate information about the causal variables in the domain, as measured in the post-assessment of domain specific knowledge.

Progress 10/20/16 to 09/01/21

Outputs
Target Audience:Target audience includes students, school districts, school teachers, science educators, college educators, and policymakers. Given that this project involves college students as participants, the conceptual information and preliminary results of this project have been shared with the students who participated in the pilot studies. Changes/Problems:The present project primarily aims to investigate college students' scientific processing skills and to examine different instructional approaches that are hypothesized to facilitate the acquisition of key scientific reasoning strategies. A critical first step involves developing tasks and procedures that can be used to effectively assess college students' scientific thinking skills/strategies in formulating hypotheses, designing valid experiments, evaluating evidence and interpreting data. We plan to complete in the process of further developing and piloting various tasks that are appropriate for presenting to college students in the classroom setting in the coming year. The scope of the present project will be expanded to examine possible changes in students' scientific thinking from their first to fourth year in college, and possible changing patterns in different majors. The results will have significant implications for possible impacts of college education on students' scientific thinking skills, and for designing curriculum and planning classroom activities and programs in college education. What opportunities for training and professional development has the project provided?Given that one major goal of the present project is to examine how college students engage in scientific thinking, this project provides an unique training opportunity to facilitate college students' scientific processing skills. The purposes and outcomes of the pilot studies were shared with those who participated in the pilots. How have the results been disseminated to communities of interest?Given that no final results of this project have been achieved, no information has been disseminated to the target audience (e.g., teachers, science educators, school districts, and policymakers). However, preliminary piloting results have been shared with the college students who participated in the pilot studies. What do you plan to do during the next reporting period to accomplish the goals?With the continued Hatch project, we plan to continue the data collection process with an expanded scope of the current studies. We will also plan to further develop and revise experimental materials, training instruction, experimental procedures for effectively assessing college students' scientific thinking skills, the acquisition of scientific thinking during their college years, and the possible different scientific thinking acquisition patterns across different majors. When additional piloting studies are completed and additional experimental materials and procedure are finalized, we plan to conduct the formal experiments and examine students' scientific thinking in their first and fourth years in college, and in different majors (i.e., humanities, social science, and natural science). Data collection and analyses will be performed in the coming year.

Impacts
What was accomplished under these goals? It is critical to develop experimental materials that can effectively assess college students' scientific thinking in a classroom setting. We have developed experimental materials for examining college students' scientific processing skills and knowledge, and have conducted three pilot three classroom experiments/assessments of college students' scientific thinking. We have also analyzed piloting data previously collected to evaluate the appropriateness of the experimental materials and procedure in classroom settings. Based on the results of the piloting, we have further revised the experimental materials, formats, and procedures. Due to the pandemic, the formal data collection for the proposed studies has been interrupted. In the coming year, we plan to expand the scope of this project by examining college students' scientific thinking to address the issues of changes in their scientific reasoning during their college years. We will plan to collect data examining students' scientific reasoning in their first and fourth year in college, and in different majors (i.e., humanities, social science, and natural science).

Publications

Progress 10/01/19 to 09/30/20

Outputs
Target Audience:Target audience includes students, school districts, school teachers, science educators, college educators, and policymakers. Given that this project involves college students as participants, the conceptual information of this project has been shared with the students who participated in the pilot study. Changes/Problems:The present project primarily aims to investigate college students' scientific processing skills and to examine different instructional approaches that are hypothesized to facilitate the acquisition of key scientific reasoning strategies. A critical first step involves developing tasks that can be used to effectively assess college students' scientific thinking skills/strategies in formulating hypotheses, designing valid experiments, evaluating evidence and interpreting data. We are in the process of developing and piloting various tasks that are appropriate for presenting to college students in the classroom setting. The scope of the present project will be expanded to examine possible changes in students' scientific thinking from their first to fourth year in college, and possible changing patterns in different majors. The results will have significant implications for possible impacts of college education on students' scientific thinking skills, and for designing curriculum and planning classroom activities and programs in college education. What opportunities for training and professional development has the project provided?Given that one major goal of the present project is to examine how college students engage in scientific thinking, this project provides an unique training opportunity to facilitate college students' scientific processing skills. The purposes and outcomes of the pilot study were shared with those who participated in the pilot. How have the results been disseminated to communities of interest?Given that no final results of this project have been achieved, no information has been disseminated to the target audience (e.g., teachers, science educators, school districts, and policymakers). However, preliminary piloting results have been shared with the students who participated in the pilot studies. What do you plan to do during the next reporting period to accomplish the goals?We plan to continue developing and revising experimental materials, training instruction, experimental procedures for effectively assessing college students' scientific thinking skills, and further piloting on classroom experiments. When additional piloting studies are completed and the experimental materials and procedure are finalized, we plan to conduct the formal experiments and examine students' scientific thinking in their first and fourth years in college, and in different majors (i.e., humanities, social science, and natural science). Data collection and analyses will be performed in the coming year.

Impacts
What was accomplished under these goals? It is critical to develop experimental materials that can effectively assess college students' scientific thinking in a classroom setting. We have further developed experimental materials for examining college students' scientific processing skills and knowledge. Due to the pandemic, the data collection for further pilot studies has been interrupted. During the reporting period, we further analyzed piloting data previously collected to evaluate the appropriateness of the experimental materials and procedure in classroom settings. In the coming year, we plan to expand the scope of this project by examining college students' scientific thinking to address the issues of changes in their scientific reasoning during their college years. We will plan to collect data examining students' scientific reasoning in their first and fourth year in college, and in different majors (i.e., humanities, social science, and natural science).

Publications

Progress 10/01/18 to 09/30/19

Outputs
Target Audience:Target audience includes students, school districts, school teachers, science educators, and policymakers. Given that this project involves college students as participants, the conceptual information of this project has been shared with the students who participated in the pilot study. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Given that one major goal of the present project is to examine how college students engage in scientific thinking, this project provides an unique training opportunity to facilitate college students' scientific processing skills because the purposes and outcomes of the pilot study were shared with those who participated in the pilot. How have the results been disseminated to communities of interest?Given that no final results of this project have been achieved, no information has been disseminated to the target audience (e.g., teachers, science educators, school districts, and policymakers). However, preliminary results have been shared with the students who participated in the pilot study. What do you plan to do during the next reporting period to accomplish the goals?Before formal experiments are conducted, we plan to continue developing and revising experimental materials, training instruction, experimental procedures for assessing college students' scientific thinking skills, and further piloting on classroom experiments.

Impacts
What was accomplished under these goals? We have developed additional experimental materials for examining college students' scientific processing skills and knowledge, and have further piloted the classroom experiment/assessment of college students' scientific thinking. During the reporting period, we have collected data for a pilot study to evaluate the appropriateness of the revised experimental materials and procedure. Based on the results of the piloting, we are further revising the experimental materials, formats, and procedures, and further piloting will be conducted. Given that no formal experiments have been conducted for this project, no formal data analyses have been performed, and no key outcomes or findings have been achieved during this reporting period.

Publications

Progress 10/01/17 to 09/30/18

Outputs
Target Audience:Target audience includes students, school districts, school teachers, science educators, as well as policymakers. Given that this project involves college students as participants, the conceptual information of this project has been shared with the students who participated in the pilot study. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest?No final results of this project have been obtained given that this is still at the piloting stage. However, we are planning to share preliminary and final results of the proposed project with instructors at all levels of education, college students, science educators, school districts, and policymakers. What do you plan to do during the next reporting period to accomplish the goals?Before data collection begins, we plan to continue developing and revising experimental materials for assessing college students' scientific thinking skills, and further conduct piloting classroom experiments.

Impacts
What was accomplished under these goals? We have further developed experimental materials for assessing college students' scientific processing skills and knowledge, and have further piloting the classroom experiment/assessment of college students' scientific thinking. Based on the results of the piloting, we are revising the experimental materials, formats, and procedures, and further piloting will be conducted.

Publications

Progress 10/20/16 to 09/30/17

Outputs
Target Audience:Target audience includes students, school districts, school teachers, science educators, as well as policymakers. Given that this project involves college students as participants, the conceptual information of this project has been shared with the students who participated in the pilot study. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?Given that this project involves college students as participants, after the classroom experiment, the conceptual information of this project has been shared in the classes, and students thus gain domain-general scientific thinking skills and knowledge through the participation in the study. The final results of the proposed project will eventually be shared with instructors at all levels of education, college students, science educators, school districts, and policymakers. How have the results been disseminated to communities of interest? Nothing Reported What do you plan to do during the next reporting period to accomplish the goals?We plan to further develop experimental materials for assessing college students' scientific reasoning skills and knowledge, and further conduct piloting classroom experiments before data collection for this project begins.

Impacts
What was accomplished under these goals? We have further developed experimental materials for assessing college students' scientific reasoning skills and knowledge, and have started piloting the classroom experiment/assessment of college students' scientific thinking.

Publications