Recipient Organization
TEXAS A&M UNIVERSITY
750 AGRONOMY RD STE 2701
COLLEGE STATION,TX 77843-0001
Performing Department
Agricultural Leadership, Education & Communication
Non Technical Summary
This project is designed to testinnovative approaches to improveprogram and project evaluation. It includes standard evaluation processes, but expands the approachto include community values, active learning, and leadership development to increase the viability of transfer of new knowledge from scientists to community stakeholders.
Animal Health Component
50%
Research Effort Categories
Basic
(N/A)
Applied
50%
Developmental
50%
Goals / Objectives
This Hatch project is designed to refine and test an EVAL framework to enhance and promote trans- and multi-disciplinary thinking and practice. The EVAL framework was recently developed with colleagues in the College of Agriculture and Life Science as a translational collaboration (bench to community) innovation in advancing human health using food choices to decrease the cost of healthcare. This framework will be adapted and tested in at least three contexts: (a) interregional technology transfer and adoption of agroecology; (b) health disparity research, and (c) underrepresented populations in STEM in collaboration with the College of Engineering. The expected outcomes include the testing of this framework for evaluating complex projects, incorporating values and opinion leadership, and to promote project sustainability and impact for evidenced-based decision making through engaged scholarship.For the last half century, social and behavioral scientists have argued that their disciplines should engage in research to help solve practical problems in communities (Western, 2019). But doing so has proved difficult. How do we join forces with bench scientists to contribute important evidence-based solutions for policy-makers, practitioners, university researchers, and other stakeholders within communities? This project will refine and test pathways, processes, or intervening variables that form causal mechanisms to measure innovation and change. Causal mechanisms include the choices and capacities which lead to regular patterns of social behavior that are usually hidden and sensitive to variation in context (Pawson & Tilley, 1997; Astbury & Leeuw, 2010). There is a need to look more deeply into the generative causation of successful programs to enable evaluators to make more credible statements on causal links between the contribution of an intervention and the observed effects (Schmitt, 2020). "For the evaluation to generate useful and reliable evidence, stakeholders should invest sufficient time to clarify conceptual issues and engage in knowledge transfer about the strengths and weaknesses of approaches in an early stage of the evaluation process" (Schmitt, 2020, p. 23). For example, if decisions that affect one part of the food system cause unexpected consequences that could impact the environment, human health and food accessibility, research related to innovation and change can inform ethical responsibility and crisis communication using a systems research design. Communication channels through opinion leaders promote transparency among stakeholders, improve communication and give voice to diverse perspectives accounting for system dynamics and complexities. Engaging with a variety of stakeholders promotes sharing of data, equitable participation, and public-private partnerships for collaboration.Purpose and ObjectivesThe purpose of this 5-year research plan is to refine and test the EVAL framework as a theory-driven impact evaluation tool that can be operationalized for translational science. Anticipated measurable impacts include: 1. increased skills to promote evaluation within organizations2. increased engagement of users and beneficiaries in evaluation processes, and3. increased collaboration of social and behavioral scientists in evidenced-based decisions.
Project Methods
Depending upon the characteristics of the object of the evaluation, evaluators may choose the level of analysis and focus the evaluation on specific behavioral mechanisms (realist evaluation) or process mechanisms (process tracing, contribution analysis, related theory of change approaches) using a case-based methodology. The initial model development was created as part of grant proposal with several scholars in the Department of Agricultural Leadership, Education, and Communications (see collaborators). From this research team, other collaborators were identified in Agroecology, Health Disparities, and STEM Pathways. The research team will serve as the key informants for entry to the community sample. To refine and test the EVAL framework in various contexts, respondents will be purposively selected based upon characteristics/criteria established by the research team from at least three evaluation projects. The sample will be drawn from beneficiaries of each context.Over the next five years I will work with research partners facilitating community needs assessments and evaluation to refine and test the framework for community engagement. Data will be collected using focus groups, participatory rural appraisal (PRA), observational field notes, and documents/photographs (Blundo-Canto, Devauz-Spatarakis, Mathé, Faure, & Cerdan, 2020; Dooley, Dobbins, & Edgar, 2018; Dooley, 2007). Pseudonyms or respondent codes will be used to ensure confidentiality.The protocol development will be based on diffusion of innovation and change theories. Multiple data sources and varying perspectives will be used in analyzing the data based upon the theoretical framework (i.e. Rogers, 1995). I will use detailed field notes to determine trends in the data from the varying perspectives. "This comparison of the data generates theoretical properties of the category...Thus the process of constant comparison stimulates thought that leads to both descriptive and explanatory categories" (Lincoln & Guba, 1985, p. 341). I will conduct a debriefing with the key informants to define categories based on overlying themes in the data. We will draw upon the principles of contribution analysis (Mayne, 2012) and participatory impact pathway analysis (Douthwaite et al., 2007) for a participatory construction and validation of the impact pathway. The results will be disseminated through collaborative seminars, workshops, and publications.