Recipient Organization
UNIVERSITY OF FLORIDA
G022 MCCARTY HALL
GAINESVILLE,FL 32611
Performing Department
Gulf Coast Research and Education Center
Non Technical Summary
During evaluation of Extension education programs, Extension professionals predominantly use self-reported measures to capture information about outcomes of Extension programs, e.g., perceived knowledge gain, attitude, intentions to adopt practices and change behaviors, and actual practice adoption and behavior change. Most commonly, surveys or interviews are used to capture information on Extension program outcomes and these self-reported measures are subjective estimates of their own knowledge, behavior and practices. Predominately, self-reported measures rests on the assumption that self-reports represent actual knowledge or behavior of an individual.Unfortunately, there has not been a systematic inquiry of the effectiveness of using these measures within Extension program evaluation. There also has not been a focus on validating self-reported knowledge measures especially in the Extension context, with most of the research in the health care field centered on practice adoption or behavior change. A significant needs exists for validating these measures and better understanding how to potentially adapt the status quo to allow for better evaluations.
Animal Health Component
100%
Research Effort Categories
Basic
(N/A)
Applied
100%
Developmental
(N/A)
Goals / Objectives
The purpose of this study is to enhance to efficacy of Extension program evaluation across multiple program areas. To achieve this purpose, this project will achieve the following objectives:Determine the relationship between self-reported knowledge measures and actual assessments of knowledgeDetermine the effectiveness of self-reported knowledge measures for predicting behavior change.Determine the relationship between subjective and objectives measures of behavior assessment.Develop Extension evaluation tools that are effective for measuring the intended outcomes of programs.
Project Methods
We propose conducting an evaluation study to understand the validity of self-reported measures across multiple programs in the family and consumer sciences program area. We will select three different programs to enhance the generalizability of findings. First, we will identify an Extension program. We will select three Extension educators in each area delivering the program in three different counties. Upon conformation of program, the evaluation specialist will observe the selected program activities for one season to understand program goals and objectives, evaluation expectations, program theory, viable data collection techniques and build rapport with participating Extension educators. Based on these observations, we will identify indicators to capture information about selected behavior and practices and develop instruments that couple actual assessments of knowledge and behavior change with already existing self-reported measures of knowledge, aspirations, behavior change and practice adoption. For objective measure of behavior and practice change we will identify multiple indicators in consultation with program stakeholders, peer evaluation (i.e., consultation with family members and friends of participants) of behaviors reported by participants, and statistical control of various moderating variable that may threaten the validity of self-reported measures such as demographic (e.g., gender, age)and methodological variables (e.g., survey characteristics). To confirm the validity of self-reported measures, we will evaluate the relationship between actual knowledge assessment and the perceived measures to understand the accuracy of the self-reported measures. We will also evaluate the ability of the self-reported measures of knowledge and aspirations to predict behavior change. Our follow up evaluations to measure behavior change will couple surveys with self-reported measures along with actual behavior change to not only measure the relationship between behavior change measures but across the measures of evaluation included in this study. The study findings will make multiple contributions to evaluation domain of cooperative extension and human resource development. In terms of contribution to evaluation domain, our study will enhance the validity of self-reported measures commonly used by Extension professionals to document program outcomes. We will develop new tools and measures to capture actual knowledge gain, behavior or practice change. For human resource development, our project will be very rewarding to participating Extension educators, because it will provide real research experience along with evaluation capacity building by working with evaluation professionals. As a result of this study, the Extension educators will gain experience in planning of evaluation, selection of evaluation questions and their respective indicators, development of program theory (e.g. logic model), evaluation instrument development, data collection, analysis and interpretation.