Source: UNIV OF WISCONSIN submitted to NRP
DEVELOPMENT AND IMPLEMENTATION OF AN ECONOMICALLY VIABLE COMPUTER VISION SYSTEM TO MONITOR AND CONTROL METABOLIC DISORDERS IN DAIRY COWS
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
ACTIVE
Funding Source
AFRI COMPETITIVE GRANT
Reporting Frequency
Annual
Accession No.
1030367
Grant No.
2023-68014-39821
Cumulative Award Amt.
$1,000,000.00
Proposal No.
2022-10741
Multistate No.
(N/A)
Project Start Date
May 1, 2023
Project End Date
Feb 28, 2027
Grant Year
2023
Program Code
[A1261]- Inter-Disciplinary Engagement in Animal Systems
Recipient Organization
UNIV OF WISCONSIN
21 N PARK ST STE 6401
MADISON,WI 53715-1218
Performing Department
(N/A)
Non Technical Summary
One of the challenges facing dairy farmers is monitoring the health of their cows during the transition period before and after calving. One reason for this challenge is the lack of integrated cow-level information among the different technologies used to monitor the cows. While advanced technology such as artificial intelligence (AI) has been proposed to address this challenge, its cost-effectiveness and feasibility for early detection of peripartum diseases in dairy cows have not been evaluated. Additionally, the application of AI in the livestock sector has not been widely disseminated to potential users. In this project, we aim to develop a computer vision system that combines body shape and feeding behavior data to improve the health and welfare of cows on farms. We will evaluate the economic impact on farms and assess consumer willingness-to-pay for products with improved animal health and welfare. This project will also include outreach activities to educate students and stakeholders on the use of AI and technology in livestock. Our goal is to create a powerful predictive model that accurately detects peripartum diseases in dairy cows in real-time and to evaluate its economic and societal impact.
Animal Health Component
35%
Research Effort Categories
Basic
30%
Applied
35%
Developmental
35%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
3073499310065%
3083499209020%
4023499202015%
Knowledge Area
308 - Improved Animal Products (Before Harvest); 307 - Animal Management Systems; 402 - Engineering Systems and Equipment;

Subject Of Investigation
3499 - Dairy cattle, general/other;

Field Of Science
2090 - Statistics, econometrics, and biometrics; 2020 - Engineering; 3100 - Management;
Goals / Objectives
We will address the Inter-Disciplinary Engagement in Animalimplementing management tools to individually monitor transition dairy cows is the lack of integratedcow-level information among sensor technologies. Even though cutting-edge technology, such asartificial intelligence (AI) systems, has been extensivelyproposed, its cost-effectiveness, feasibility,and integration as a tool for early detection of peripartum diseases in dairy cows has not been criticallyevaluated or developed into powerful predictive analytics for optimized management decisions.Moreover, in the livestock sector, this emerging field of AI has not yet been widely translated throughextension and outreach programs to better inform and educate potential users. In this project, we willdevelopand implement a computer vision system that integrates novel phenotypes from body shapeand feeding behavior to improve animal health and welfare in livestock farms. We will alsoinvestigate both farm-level economic impacts and the complementary assessment of consumerwillingness-to-pay forproducts with improved animal health and welfare. Our goal is to createpowerful predictive modeling frameworks for accurate, real-time detection ofperipartum diseases indairy cows and to evaluate its economic and societal impact. Such goals will complement novelextension and outreach activitiesproposed to educate new generations of students andstakeholders on AI, data, and technology for livestock. The integration of novel phenotypesthroughAI technology is critical to advance farm management decisions when volatile profit margins,labor shortage, and suboptimal animal welfare aredetrimental factors constraining farmsustainability.We assembled a team of experts in dairy cattle management and welfare, economics, andcomputerscience. We will develop and deploy an integrated computer vision system to monitor healthproblems in dairy operations and an extension program toeducate farmers, stakeholders, and studentsfor the new era of digital solutions by achieving four aims:Aim 1. Develop a computer vision system that integrates animal recognition with real-timemonitoring of feeding behavior and body tissue mobilization.Aim 2. Integrate new features acquired using a computer vision system with other animal-levelinformation into predictive analytical tools forearly detection of health issues in dairycows during the transition period.Aim 3. Create a novel extension program on artificial intelligence for animal farming toeducate the next generation of students, farmers, andindustry stakeholders.Aim 4. Evaluate the economic impact of technology driven decisions and consumerwillingness-to-pay for potential improved animal healthand welfare outcomes throughuse of the proposed computer vision system.
Project Methods
Aim1.A RFID-camera systems will be installed at the UW-Madison Research Farms (Arlington-WI and Marshfield-WI) in three barn locations: (1) above the water tank in the dry cow pens; (2) above the water tank in the close-up pens; and (3) at the exits of the milking parlor. The CVS will be developed to extract 3D image-features related to cows' body shape. Our objective is to monitor a total of 1,000 dairy cows from -60 to +60 DRTC. This sample size will provide enough statistical power to detect even low correlations between image-based features and BCS and animal performance variables. The CVS will have depth cameras from Intel® RealSense™ Depth Camera D455, already tested by our research group, that will acquire a top-down view infrared and depth images from the cows' dorsal area. We will collect BCS from all cows to train a deep neural network to perform BCS classification that will be used as a predictor of metabolic diseases associated with negative energy balance and will serve as a control measurement of body shape changes. Subsequently, two important steps will be performed: feature extraction, and model development. Feature extraction will be implemented using two approaches: 1) biological features, and 2) computational features. The biological features, here called biometric body measurements, are known to be associated with BW and shape, such as dorsal area, dorsal width, body volume, eccentricity, and Fourier shape descriptors. The computational features will be extracted using the feature maps of the pre-trained CNN for animal body segmentation. Additionally, we will use the features of the last dense layer of a pre-trained CNN used to classify body condition score. The output of the proposed deep neural networks will be used in combination with the biological features as image-based predictors, and in combination with other covariates to early detect health issues associated with negative energy balance.Aim 2.Two groups of covariate data sets will be evaluated.First is the PREPARTUM GROUP:Set 0) use of BCS generated using the CVS;Set 1)variables from body shape (biological and computational) obtained from CVS during the prepartum period;Set 2) variables from feeding behavior obtained from CVS during the prepartum period;Set 3) Set 1 + Set 2;Set 4)Set 1 + Set 2 + cow records from herd management software. Second is the POSTPARTUM GROUP:Set 5)Set 1 +variables from body shape obtained from CVS in the postpartum until the diagnostic of health problem; andSet 6)Set 3 + variables from body shape and feeding behavior obtained from CVS in the postpartum until the diagnostic of health problem. For the predictive analyses of health problems associated with NEB, data regarding individual cows' health events such as ketosis, endometritis, abomasum displacement, and milk fever will be used, and the time series (body shape features, feeding behavior and cow records) measured before the event will be used to create predictive models. Different lengths of time series and varying intervals from the last predictor measurement to the health event will be evaluated to determine the optimum data to carry relevant signal and to assess how early health problems can be detected accurately. Staff-recorded reports of health problems will be validated as a true health event using proof of diagnosis or treatment. The longitudinal data from all sources will be tracked until the health event occurred. We will explore the use of Recurrent Neural Networks and Logistic Regression for early prediction of health events, such as ketosis, abomasum displacement, clinical mastitis, and metritis. Lastly, we will use individual cow image sequences acquired during the prepartum period (-60 to 0 DRTC) to predict potential health problems after calving.Aim 3.We will populate an independent and unbiased web portal, openly and freely available, within the University of Wisconsin-Madison Dairy Management to list all the digital technologies available for dairy farming. We will tap on Dr. Dorea's class materials of AN/DYSCI 875 that has already a long list of technologies including activity sensors, boluses, computer vision, sound recognition, etc. and build upon it to maintain the world's most comprehensive and the most up-to-date list of technologies available in the dairy industry. For each technology, there will be a systematic description of what the technology does, how it does it, how much it cost, pictures and videos (as available), and contact information. Once the site is launched, we will publish an article in the Hoard's Dairyman magazine about the existence of the web portal. Our outreach specialist will be in charge to maintain the list up-to-date, clearly organized, and completely searchable. We will take every contact opportunity with farmers and other stakeholders (e.g., extension meetings) to highlight and point to the audience to the web portal. As part of our awareness efforts, we will launch a monthly podcast, "The Digital Livestock," to discuss technologies in the livestock world.Based on our experience of creating AI tools for outreach and participating on K-12 demonstrations, open-science events, and UW-Madison Science Expedition, we will expand and increase our participation to dairy farming stakeholders including farm managers, employees, consultants, extension educators, vendors, and others. We will participate in all possible hands-on events such as yearly WI state fair and the World Dairy Expo in Madison-WI with a booth to demonstrate precision livestock technologies.We will develop online training modules addressed to farmers, extension educators, and farm consultants that will deal with three aspects of farm knowledge, all of them related to improve their readiness to adopt new technologies in the realm of digital agriculture and precision dairy farming: 1) installations, 2) personnel, and 3) data management. The training module about installations will deal on how new or improved installations planned at the farm could be made more technology friendly by including connectivity hubs, outlets, or strategic lights. Similarly, the personnel module will deal on how the farmer could support technological education of existing or new employees and will include not only resources available online and everywhere else, but also contact of knowledgeable professionals who could respond specific farm questions.Aim 4.We will develop a dynamic optimization and simulation model of a dairy herd. The model will be constructed using a hierarchical Markov chain process in monthly steps/stages. Each cow will be assigned a state based on production characteristics, and the model's aim will be to maximize the net present value (NPV) of a cow in each state-stage. During the transition period, cows will have a risk for transition diseases based on the reported incidences from the participating farms. Transition diseases will be classified as uterine (UTD) and non-uterine diseases (NUTD). We will evaluate how disease risk and losses would change if changes in management decisions due to the use of the CVS and early identification of high-risk cows were implemented.To assess the impact of the use of CVS and its potential impacts, we will apply the Becker-DeGroot-Marschak auction (BDM) approach to elicit willingness-to-pay (WTP) for the four lettuce categories.Under the BDM auction procedure, subjects individually submit sealed bids for a good. Next, a random number or price is drawn from a pre-specified distribution. Individuals whose bid is greater than the randomly drawn price "win" the auction and can purchase the good at the randomly drawn price. We will conduct separate experiments for beverage milk products and cheese to determine whether WTP is affected differently based on the product made from farm milk.Econometric models including participant demographic characteristics will then be used to assess correlates of WTP estimates.

Progress 05/01/23 to 04/30/24

Outputs
Target Audience:During the reporting period, the PI (Dorea) and the co-PIs (Cabrera, Nicholson, Van Os, and Lee) began reaching target audiences through research presentations. The PI and Co-PIs of this project reached agricultural professionals, scientists, and researchers working with high-throughput phenotyping, computer vision, precision livestock farming, genomic applications, and farm management. We discussed the challenges and opportunities for technology adoption, data analytics and integration for dairy farming. Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided?The PI is directly supervising a Postdoctoral fellow and a PhD student and co-supervising a Postdoctoral fellow with Drs. Cabrera and Nicholson. In addition, an undergraduate student has been recruited to assist with on-farm data collection and data processing. How have the results been disseminated to communities of interest?There has been significant dissemination of data to the research community during the 2023 reporting.Results are being disseminated through national and international conferences, scientific publications, and outreach activities and our extension platform developed as part of this proposal to inform farmers, industry, students, and researchers about sensing technology and artificial intelligence for animal farming.Abstracts were presented in person at the American Dairy Science Association annual meeting, US Precision Livestock Farming Conference (USPLF), American Society of Animal Science annual meeting, and many other conferences listed below. Additionally, two papers have been published, and four additional manuscripts are in progress for the 2024 reporting year. We published one image dataset for animal identification through OSF. What do you plan to do during the next reporting period to accomplish the goals?We will continue to work towards the goals. We will actively develop the extension and outreach programs and ensure that the research articles planned for 2024 are published and disseminated through conferences and research symposia.

Impacts
What was accomplished under these goals? (1) We have collected data on more than 300 cows during transition period as proposed in objective 1 and 2. Blood samples were collected for analyses of plasma non-esterified fatty acids (NEFA) and beta-hydroxybutyrate (BHBA). A total of 2.5 million depth and infrared images were collected for objectives 1 and 2. The images were all pre-processed and analyzed for Body Condition Score (BCS) and subclinical ketosis detection. Behavior data (lying time, rumination time, standing time, visits at the feed bunk, number of meals, meal duration, and cow records including days in milk, parity, and health records on previous lactation. These images are also being used to build models for animal identification and combined with cow history for disease prediction. We have participated in the UW ScienceExpedition and Wisconsin Science Festival in 2023, showcasing the project and the use of the technology and artificial intelligence in animal farming for K-12 and Wisconsin community: (1)2023 Wisconsin Science Festival Expeditions: Artificial Intelligence for Animal Farming, 988 participants attended: K-12 students from 15 schools in Wisconsin; (2) 2023 UW-Science Expeditions: Digital Technologies for Animal Monitoring - 850 participants attended: K-12 and general public.Our stations reached a total of 1,838 participants on these two events. As part of this project, we created the Smart Farm Hub platform (https://dairyintelligence-staging.webhosting.cals.wisc.edu), where informationwill start to be archived and disseminated. The Instagram account (@smartfarmhub) was also created to support the website and be used to publish the "shorts" of interviews, full interviews, students spotlight profile, and industry spotlight, with experts about technology in animal farming. The first sequence of publications will encompass the interviews collected during the 46thADSA Discover Conference. For the following year, a sequence of podcasts and shorts will be scheduled and release as part of the extension and outreach program. Additionally, we will continue engaging with K-12 through events as UW Science Expeditions and Wisconsin Science Festival.

Publications

  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Ferreira, R. E. P., M. C. Ferris, and J. R. R. Dorea. 2023. Optimizing training sets for individual identification of dairy cows. J. Dairy Sci. 106:429. Annual Meeting ADSA, Ottawa, Canada.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Dorea, J. R. R. Artificial intelligence for livestock systems. 2022. J. Dairy Sci. 105:98. Annual Meeting ADSA, Kansas City-MO, USA.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Artificial Intelligence for farm management and precision phenotyping. 46th ADSA Discover Conference: Milking the Data  Value Driven Dairy Farming. Chicago-IL, May 7th, 2024.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Transforming Dairy Farm Management with the Power of Artificial Intelligence. IDF World Dairy Summit. Chicago-IL, October 15th, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Digital technologies and Machine learning: A new way to look at novel traits at spatial and temporal dimensions. Annual Meeting of American Association of Animal Science (ASAS), Albuquerque-NM, July 22nd, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Precision Livestock Farming for Optimal Management. SIAM, Moroccan International Agricultural Show. Meknes, Morocco. May 4th, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Artificial Intelligence and Machine Learning for Agriculture. Summer School on Data Science. Fundacao Getulio Vargas. Rio de Janeiro, Brazil, Jan 24th, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Leveraging Artificial Intelligence to Optimize Farm Management Decisions. National Agricultural Producers Data Cooperative. University of Nebraska-Lincoln. September 19th, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Connectivity for Rural Areas and Sensing Technology Deployment. FCC Precision Ag Taskforce - Connectivity Demand. February 9th, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Ferreira, R. E. P., T. Bresolin, P. L. J. Monteiro, M. C. Wiltbank, and J. R. R. Dorea. 2023. Using computer vision to predict cyclicity of dairy cows during the transition period through 3D body surface images. J. Dairy Sci. 106:428. Annual Meeting ADSA, Ottawa, Canada.
  • Type: Conference Papers and Presentations Status: Accepted Year Published: 2023 Citation: Computer vision and Machine Learning for Optimized Farm Management Decisions. CALS Data Science Showcase. November 1st, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: AI from Foundations to Applications. Exploring Artificial Intelligence @ UWMadison. June 30th, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Shaping Futures with Data and Computing. Research Bazaar: Closing Panel. February 3rd, 2023.
  • Type: Websites Status: Published Year Published: 2023 Citation: Smart Farm Hub is the extension and outreach platform created for the project that will disseminate the results and provide educational materials. Website: https://dairyintelligence-staging.webhosting.cals.wisc.edu Instagram: @smartfarmhub
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Ferreira, R. E. P., Y. J. Lee., J. R. R. Dorea. 2023. Using pseudo-labeling to improve performance of deep neural networks for animal identification. Scientific Reports. 13:13875. https://doi.org/10.1038/s41598-023-40977-x
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Menezes, G. L., T. Bresolin, R. E. P. Ferreira, H. T. Holdorf, H. M. White, S. I. A. Apelo, J. R. R. Dorea. 2023. Near-infrared spectroscopy analysis of blood plasma for predicting nonesterified fatty acid concentrations in dairy cows. JDS Communication. https://doi.org/10.3168/jdsc.2023-0458
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Ferreira, R. E. P., and J. R. R. Dorea. Cloud computing to automate phenotype collection and data analyses in dairy systems. Proceedings of the US Precision Livestock Farming, p.131. Knoxville-TN.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Bresolin, T., R. E. P. Ferreira, G. J. M. Rosa, and J. R. R. Dorea. 2023. Computer vision on the edge: A computing framework for high-throughput phenotyping in livestock operations. Proceedings of the US Precision Livestock Farming, p.151. Knoxville-TN.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Menezes, G. L., A. Negreiro, R. E. P. Ferreira, and J. R. R. Dorea. 2023. Identifying dairy cows using body surface keypoints through supervised machine learning. Proceedings of the US Precision Livestock Farming, p.360. Knoxville-TN.