Source: MICHIGAN STATE UNIV submitted to NRP
FACT-CIN: A COORDINATED INNOVATION NETWORK FOR ADVANCING COMPUTER VISION IN PRECISION LIVESTOCK FARMING
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
ACTIVE
Funding Source
AFRI COMPETITIVE GRANT
Reporting Frequency
Annual
Accession No.
1025630
Grant No.
2021-67021-34150
Cumulative Award Amt.
$1,000,000.00
Proposal No.
2020-08843
Multistate No.
(N/A)
Project Start Date
Mar 1, 2021
Project End Date
Feb 28, 2026
Grant Year
2021
Program Code
[A1541]- Food and Agriculture Cyberinformatics and Tools
Recipient Organization
MICHIGAN STATE UNIV
(N/A)
EAST LANSING,MI 48824
Performing Department
ANIMAL SCIENCE
Non Technical Summary
Precision livestock farming (PLF) consists of the application of technology within the animal space to allow automated and real-time decision making at the individual animal level in livestock farming. Despite its great promise to increase profitability and productivity, on-farm adoption of PLF is slow due to technology costs and the limited knowledge base currently focused on developing and implementing PLF in livestock agriculture. A low-cost technology that holds great promise for further advancing precision livestock farming is computer vision (CV). Computer vision enables task automation by using computers to understand and extract important features of a physical system from digital images or videos. In this proposal a synergistic team of researchers from four leading institutions in the fields of Animal Science, Computer Science and Engineering will seek to advance development of CV in PLF through two main objectives. 1) Creating and releasing reference datasets and benchmarking data to facilitate the development of computer vision applications that address key challenges in precision livestock farming. 2) Building a coordinated innovation network (CIN) of stakeholders, researchers, and students to speed development of computer vision applications in precision livestock farming. Reference datasets and benchmarking data will be generated at four different institutions from pigs and cattle, including annotated images and ground truth data for animal identification and detection of various behavior. Further, baseline performance results will be generated by applying existing analysis methods to these datasets. An analytical challenge series organized around the reference datasets and a webinar series on CV and PLF that will attract a broad base of researchers from animal sciences and veterinary medicine to computer sciences and engineering to build the CIN.
Animal Health Component
100%
Research Effort Categories
Basic
(N/A)
Applied
100%
Developmental
(N/A)
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
3073410208030%
3073510208030%
3073499310020%
3073599310020%
Knowledge Area
307 - Animal Management Systems;

Subject Of Investigation
3499 - Dairy cattle, general/other; 3510 - Swine, live animal; 3410 - Dairy cattle, live animal; 3599 - Swine, general/other;

Field Of Science
2080 - Mathematics and computer sciences; 3100 - Management;
Goals / Objectives
This project has two objectives and several subobjectives detaled below. Objective 1: Generate reference datasets and benchmarking data for facilitating the development of computer vision applications that address key challenges in precision livestock farming.Sub-objective 1a: Generate reference datasets for testing animal identification algorithms. Sub-objective 1b: Generate and distribute reference data for quantifying behavior using CV. Sub-objective 1c: Provide a set of baseline performance results from applying existing analysis algorithms to the data generated in Sub-objectives 1a and 1b.Objective 2: Build a coordinated innovation network of stakeholders, researchers, and students to develop computer vision applications in precision livestock farming.Sub-objective 2a. Organize a "Computer Vision-for-PLF challenge series". Sub-objective 2b. Host a webinar series on translational topics in Computer Vision and PLF
Project Methods
Objective 1: Generate reference datasets and benchmarking data for facilitating the development of Computer vision applications that address key challenges in precision livestock farming.Sub-objective 1a: Generate reference datasets for testing animal identification algorithmsThe general problem that we will address in this sub-objective is the generation of reference datasets for animal detection, identification, and tracking from imagery data. The overall challenges that we intend to address with these data are the following:Detect individual animals within a picture of a group of animalsAssign an identity to each detected animalTrack individuals through sequences maintaining their identity despite contact and occlusionsRe-identify each animal over an extended period of time using different sensors if necessary.We will accomplish this through a combination of re-annotation of existing dataset and new data recording. The data will include: nursery pigs, growth finishing pigs, lactating dairy cows and dairy cattle. These datasets will include digitial and depth images and videos of animals throughut their production cycle as well as positive identification of each animal through direct human observation or automatic RFID systems.Sub-objective 1b: Generate and distribute reference data for quantifying behavior using CVUnder this sub-objective, we will focus on the generation of video to automatically detect specific behaviors in pigs and cattle. The behaviors and activities will include:post-mixing pig-pig aggression. These will include damaging aggression such as attacks and bites and non-damaging aggression, such as inverse parallel pressing.Non-damaging behavioral interaction to displace animals from feeders, including those that imply physical contact, such as mounting, head knocking, pushing, mounting, and those that don't include physical contact as described below.Feeding and drinking activity, including growing and nursing animalsPlay and other positive interactions between animalsThese raw data will come in the form of video recordings. Each set of video recordings will be annotated for more than one behavior. For instance: In one video, aggression, play, and drinker and feeder use could be combined. Video annotation will be performed by trained experts. These data will partially overlap with data for Sub-objective 1a. Additionally, under this objective we will include recordings of farrowing sows.Sub-objective 1c: Provide a set of baseline performance results from applying existing analysis algorithms to the data generated in Sub-objectives 1a and 1b We will provide baseline algorithms for each of the tasks proposed in Sub-objectives 1a and 1b and the associated analytical challenges in Sub-objective 2a. These algorithms will be described and released along with datasets and challenges in open access publications. Github pages will be used to manage and distribute each baseline algorithm, with codes written in Python using PyTorch or Tensorflow deep learning libraries. Trained networks will be published using the Open Neural Network Exchange format to allow researchers to integrate them into a variety of other neural network libraries like Microsoft CNTK, MATLAB, and Caffe2.The computer vision algorithms that apply to the tasks listed in Sub-objectives 1a and 1b can generally be broken up into: 1) Multiple Instance Detection and Pose Estimation, 2) Target Re-Identification and Long-Term Tracking, and 3) Action Recognition.Objective 2: Build a coordinated innovation network of stakeholders, researchers, and students to develop translational computer vision applications in precision livestock farming.Sub-objective 2a. Organize a "Computer Vision-for-PLF challenge series".We will generate analytic challenges centered around the data generated under Objective 1. Each challenge will address important aspects of PLF for which computer vision and machine learning could leverage our datasets to provide solutions. We expect to launch one analytical challenge per year of the project. Each analytical challenge will be organized by a committee initially composed of members of the PI team, but which will include other participants of the network over time, as described below.For each challenge multiple training datasets, consisting of annotated video and images, will be made publicly available through the open science foundation (OSF, https://osf.io/). Each challenge will have two phases. Phase 1 starts after data are publicly released and culminates in a workshop or meeting with peer-reviewed publications, and during which official winners are announced using a leaderboard. Phase 2 is an indefinite open challenge during which data on OSF remain available, and the leaderboard is maintained and updated submission of new or improved techniques gain recognition.The annotated training data will be publicly available for each challenge, while ground truth test annotations will be private. Participants must agree to the challenge rules for their test-data submission to be scored and will be asked to cite a DOI registered for each project. The OSF wiki will provide detailed data descriptions, as well as scoring criteria on test data.Some sample challenges are:1) Image-based re-identification of marked and unmarked pigs and cattle. Large multi-modal datasets will be used with animals observed from multiple viewpoints and annotated with a unique ID. 2) Video-based tracking of livestock including through occlusions and hand-off of tracks between sensors. RFID tags and other sensors will be used for ground truth annotation. 3) Automatic detection of aggression from video clips including behaviors such as reciprocal fights, single-sided attacks, and non-aggression. 4) Activity classification from video including drinking, feeding, standing, running and exploring.Sub-objective 2b. Host a webinar series on translational topics in Computer Vision and PLFWe will host a webinar series covering topics to promote understanding of and innovation in using CV for PLF applications. Webinars will combine presentation and 'unconference' discussion elements and will showcase diverse perspectives and evolving knowledge related to CV and PLF.The webinars will begin in the first year of the project and continue across the life of the grant from September to May. Initial topics will provide an overview of PLF and describe the status of research and commercial application of PLF, with particular reference to pigs and cattle . Next we will set the stage for the environment in which the CV-PLF applications must operate by inviting experts in livestock production, health, and welfare to discuss current challenges in practical production systems and provide their thoughts on how CV could be most useful in developing PLF for animal management. The webinars will then segue into more technical presentations from academic and industry experts working on a wide range CV and PLF topics. A final area of focus for the webinars will be presentations from possible funders of CV-PLF research, including USDA-NIFA, to describe current opportunities. Each year, the cycle will repeat, with new speakers and perspectives to reflect the rapid innovation, ongoing adoption of PLF in practice, and new challenges that will arise over the lifetime of the project.Speaker presentations will be 30 minutes in length, followed by moderated 'unconference'-style discussions that will be arranged around questions submitted by the audience and conference organizers and the goals of the registrants. Members of the project team, including students and postdocs, will be tasked with generating questions or points to raise during the discussion to ensure that dialog is stimulated. A member of the project team will moderate the discussion to ensure equitable and inclusive discussion.

Progress 03/01/24 to 02/28/25

Outputs
Target Audience:Animal Scientists, animal breeders, ethologists, engineers, data analysts and computer scientists interested in the application of computer vision in animal production systems were reached during the current reporting period. Changes/Problems:At Iowa State, recruiting a data scientist was difficult as university salaries for technical personnel in this area were not competitive with private sector. A post-doctoral researcher took over some of the work that was planned for data scientists, such as data maintenance and sharing. What opportunities for training and professional development has the project provided?In total, 4 postdoctoral researchers, 5 graduate students and 2 undergraduate students were trained as part of the project in the current reporting year. Additional students benefited via coursework and class projects based on our findings and datasets. At Iowa State University, two undergraduate students, three graduate students and one postdoctoral researcher were trained in image collection, image annotation and image analyses. Graduate students presented their work at professional conferences. A computer vision study group was established at ISU led by Post Doctoral Trainee Ye Bi, and it includes 4 faculty members from Animal Science and Computer Engineering and 5 graduate students. The group meets regularly to discuss projects, current literature and to coordinate data sharing.At Michigan State University, postdoc Bhujel is receiving training in on farm research as well as learning about animal behavior and welfare applications of PLF. Dr. Morris has weekly in-person meetings with Dr. Bhujel to discuss plans and progress in research. At bi-weekly intervals, Dr. Bhujel presents his progress during the Smart Sensor Lab meetings. Once per month, Dr. Bhujel presents his work to PI-Siegford and Co-PIs Morris and Benjamin and discusses plans. Dr. Bhujel has submitted a paper to USPLF and has been working on a research proposal. At University of Nebraska-Lincoln, one PhD student and a postdoc were engaged in the project. Postdoc Sharma contributed to the development of the image-capture system, while a graduate student focused on processing both digital and depth images into point cloud representations. PhD candidate Paudel gained experience using PointNet to evaluate a model's ability to identify pigs at three different production stage. Both have been receiving training to analyze, present and publish their findings with key audiences. At KU Leuven, as part of the work completed on the project over the last year, we have trained four master students and taught this system in multiple master program courses. In the current year, one PhD student and one postdoc were engaged in research related to the project. How have the results been disseminated to communities of interest?A substantial number of conference presentations (and subsequent proceedings papers) were delivered by PIs, postdocs, and students at the European Conference on Precision Livestock Farming in September, 2024, in Bologna, Italy. The team also presented at a range of other conferences such at the Annual Conference of the American Society of Animal Science, the American Dairy Science Association Discover Conference, and the American Society of Agricultural and Biological Engineers. Invited talks at universities in the US, Brazil and Canada were also given by team PIs and our work helped inform efforts led by co-PI Norton in Europe within the CIGR International Commission of Agricultural and Biosystems Engineering. Webinars organized by co-PI Benjamin at Michigan State University continued to reach a global audience of industry, academic and government members and covered topics ranging from drone use to thermal monitoring and body worn sensors. What do you plan to do during the next reporting period to accomplish the goals?As we enter the final year of the project, our emphasis will be on publishing and sharing our findings, resources, and data. All shared datasets will be published in the Open Science Framework (OSF) and will be shared in Kaggle in "competition format". We will continue to complete the final peer-reviewed journal articles, two of which are accepted, two more have recently been submitted and another two are already in preparation. Many of the PIs and their students and postdocs will be presenting findings at the 3rd US Precision Livestock Farming Conference in June 2025 in Lincoln, Nebraska, which is being organized by co-PI Brown-Brandl. A final activity, led by co-PI Morris at Michigan State University, will be to distribute two public leaderboards and challenges.

Impacts
What was accomplished under these goals? At Iowa State University, datasets for pig tracking, keypoint detection, and lameness detection have been shared. A dataset for Pig ID detection is being created along with another examining phenotypic traits related to farrowing.At Michigan State University, co-PI Morris and postdoc Bhujel completed work on Sub-objective 1b including building and configuring a dual-pen camera system with 8 cameras for monitoring gilts including interactions at feeders. RFID tag readers were installed on the feeders for obtaining unique gilt IDs, and software was developed for recording and aligning synchronized video as well as IDs. This resulted in recording over 100 TB of video data. Under sub-objective 1c, the Michigan State team implemented, trained and tested a 4-keypoint swine detector as well as a bounding-box detector and then integrated these into a tracker. They are currently quantifying the performance of the detectors and trackers. Under sub-objective 2a, the MSU team collated and annotated a dataset of gilt daily activities, and is working towards releasing this as a computer vision challenge on Kaggle. MSU also hosted 4 more PLF webinars that are available to the public via the university website. At University of Nebraska-Lincoln, an RFID-camera system was installed above the drinkers in the pen, enabling capture of images tagged with each animal's RFID number. We tested two different models across several image resolutions to determine the most effective method for individual pig identification. Once the initial method was established, we applied PointNet to evaluate the model's ability to identify pigs at three different production stages: early, mid, and late finishing. After the initial study, we applied the best-performing model, PointNet, to classify more than 20 grow-finish pigs using approximately 800 point clouds per animal. Results demonstrated that identification during early finishing was inconsistent, with macro-averaged F1 scores ranging from 0.10 to 0.46 over 15 days. However, mid- and late-finishing pigs were identified with greater reliability. In the mid-finishing stage, the model achieved F1 scores above 0.87 for four consecutive days and maintained an F1 score of 0.81 after 15 days. In the late-finishing stage, F1 scores exceeded 0.90 and remained above 0.85 for up to 13 days. Long-term identification between mid- and late-finishing stages was also feasible, with F1 scores of 0.47 up to 44 days in the future. Performance was further improved by retraining the model with a small volume of intermediate data. These findings indicate that high-performing individual identification of pigs using point cloud data is achievable during later growth stages, particularly with strategically timed model retraining. At KU Lueven, significant progress has been made in livestock tracking in this period, with key contributions from postdoc Dong Liu who collected an original dataset covering over 40 pig pen environments. This dataset supported the development and validation of new computer vision tracking and detection methods. Specifically, a tailored pig detection algorithm with Rotated Bounding Box (RBB) was developed. A harmonized annotation protocol was developed between M3-BIORES and other collaborators from the EU-LI-PHE network. This protocol was used to create an annotated pigs and dairy cow tracking dataset, facilitating consistent and comparable research. The initial text for a scientific article on this was written, and key figures produced. Additionally, an annotation protocol using rotated bounding boxes for dairy cows was newly defined. To enable broader scientific transfer and more standard evaluation, suitable platforms for hosting a benchmark challenge were surveyed and identified, including Kaggle. A selected platform was tested with a small dataset to confirm suitability. A harmonized annotation format for challenge submissions was agreed. Based on the developments done, we have trained four master students and taught this system in multiple master program courses.

Publications

  • Type: Peer Reviewed Journal Articles Status: Published Year Published: 2025 Citation: Agha, S., Psota, E., Turner, S. P., Lewis, C. R., Steibel, J. P., & Doeschl-Wilson, A. (2025). Revealing the Hidden Social Structure of Pigs with AI-Assisted Automated Monitoring Data and Social Network Analysis. Animals, 15(7), 996.
  • Type: Peer Reviewed Journal Articles Status: Published Year Published: 2025 Citation: Paudel, S., Brown-Brandl, T., Rohrer, G. and Sharma, S.R., 2025. Deep learning algorithms to identify individual finishing pigs using 3D data. Biosystems Engineering, 255, p.104143.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Mao A, Zhu M, Huang E, Guo Z, He Z, Lyu L, Norton T, Liu K. 2024. Cross-species knowledge sharing for improved animal activity recognition with limited labelled data. In: Proceedings of 11th European Conference on Precision Livestock Farming. Bologna, Italy.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Negreiro A, Alves A, Ferreira R, Bresolin T, Menezes G, Casella E, Rosa GJM, Dorea JRR. 2024 Siamese Networks for identification of Holstein cattle during growth and across different physiological stages. In: Proceedings of 11th European Conference on Precision Livestock Farming. Bologna, Italy.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Parmiggiani A, Liu D, Norton T. 2024. Pig behaviour classification with CRNN. In: Proceedings of 11th European Conference on Precision Livestock Farming. Bologna, Italy.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Paudel S, Brown-Brandl T, Ramirez B, Viera de Sousa R, Banhazi T, Norton T. 2024. Survey results of swine production across the globe. In: Proceedings of 11th European Conference on Precision Livestock Farming. Bologna, Italy.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Rahman, M.T., Brown-Brandl, T.M., Rohrer, G.A., Sharma, S.R. and Shi, Y., 2024. Classification of Sow Postures Using Convolutional Neural Network and Depth Images. In 2024 ASABE Annual International Meeting (p. 1). American Society of Agricultural and Biological Engineers.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Shumaly, M., Park, Y, Agha, S., Pandey, S., & Steibel, J.P. 2024. Hierarchical models to account for uncertain identification in precision livestock farming and phenotyping. In: Proceedings of 11th European Conference on Precision Livestock Farming. Bologna, Italy.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Steibel JP, Siegford JM. 2024. Dyadic linear models for genetic analysis of behavioral interactions. Measuring Behaviour 2024 Conference Proceedings. May 15-17, 2024, Aberdeen Scotland. pp. 254-257. (Invited talk.)
  • Type: Conference Papers and Presentations Status: Other Year Published: 2024 Citation: Dorea JRR. 2024. Artificial Intelligence for farm management and precision phenotyping. 46th ADSA Discover Conference: Milking the Data  Value Driven Dairy Farming. Chicago-IL, May 7th, 2024.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Shumaly, M., Park, Y. Taylor, W., Pandey, S., Steibel, J.P. 2024. Biologically informed machine vision for gait pattern analysis yields novel locomotion phenotypes in pigs. Proceedings of AGBT-Ag. Student selected talk.
  • Type: Conference Papers and Presentations Status: Other Year Published: 2024 Citation: Steibel, J.P., (2024) Sensors and Images for phenotyping in Animal Breeding. SimMelhor. III Symposium in Animal Breeding. University of Vi�osa, Brasil. (Invited talk.)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Steibel, J.P., (2024) Computer vision-based phenotyping: dealing with uncertainty in animal identification. Invited Talk. Invited talk in Animal Breeding and Genetics Symposium: Genome-enabled optimization of deep phenotyping. Annual meeting of ASAS. Canada.
  • Type: Other Status: Other Year Published: 2024 Citation: Steibel, J.P. (2024) LeClerg Endowed Lecture: The integration of phenomics and genomics to investigate the genetics of competition and social behavior. University of Maryland. (Invited talk.)
  • Type: Websites Status: Published Year Published: 2025 Citation: Computer Vision in Precision Livestock Farming Webinar Series. Updated 2025. https://www.canr.msu.edu/precision-agriculture/Precision-Livestock-Farming-Webinar-Series/2024-webinar-recordings


Progress 03/01/23 to 02/29/24

Outputs
Target Audience: Animal Scientists, animal breeders, ethologists, engineers, data analysts and computer scientists interested in the application of computer vision in animal production systems. Changes/Problems:A challenge in current year was getting a PhD student to work at MSU related to to benchmarking and analytic challenges. Several PhD students started but then did not stay with the project. We transitioned to searching for a postdoc instead, and after a time intense search process have recently hired a person (Anil Bhujel) who will join us in several months. Once Dr. Bhujel is in place, he will work on the analytical challenges and benchmarking datasets to bring us up to speed here. What opportunities for training and professional development has the project provided?Three graduate students and three undergraduates received training as part of their work on this project. Several of the students have presented their work at conferences, gaining professional development opportunities related to communicating their data and networking. Additionally, the webinars (and resulting recordings which are available online) have been attended (and viewed) by graduate students and postdocs within and beyond the team, providing them with opportunities to network as well as exposure to a range of problems and approaches in computer vision to solving PLF-related problems. How have the results been disseminated to communities of interest?Invited presentations have been delivered by PDs Dorea, Rosa, Steibel, and Siegford as reported. Papers in reputable peer- reviewed scientific journals were also published. Several invited presentations and contributed presentations were delivered in several venues, including the following scientific meetings: American Society for Animal Science (ASAS), the 2nd US Precision Livestock Farming Conference, the Midwest (ASAS) and American Society of Agricultural and Biological Engineers. What do you plan to do during the next reporting period to accomplish the goals?In the coming period we will move forward in several areas. More on farm video of pigs (sows, piglets) and laying hens will be captured for Objectives 1a and 1b. The team will add at least two new curated datasets to the public repository and several publications along with them. We will begin work on benchmarking datasets. For Objective 2a, we plan to deliver at least one analytical challenge in the coming year around the available data collected and summarized during earlier reporting periods. We also plan to continue to generate novel data for challenges in successive years. Under Objective 2b, we will continue to host the webinar series through September 2024 and continue to include presenters contacted through activities of this grant (webinar participants, challenge participants, etc) in order to grow the CIN and document its impact

Impacts
What was accomplished under these goals? Multiple datasets from pigs and one from poultry were collected this year for use in Objectives 1a and 1b. These will be used to work on animal identification and activity tasks proposed in the grant. They can also in future be used for data challenges as they are made publicly available. A webinar series on computer vision in PLF (Objective 2b) was started 2 years ago and continued through this year. In the current reporting period 7 webinars were presented. This series caters to a broad and diverse audience from every continent and from several disciplines related to computer vision, genetics, engineering, and animal production. The feedback from the audience has been excellent and was used to select topics for the most recent webinars. When our group reports/presents results in conferences, we are receiving now requests of other groups to schedule presentations in the webinar series. We have recorded these webinars, and all (including from past years) are freely available at Precision Livestock Farming website at MSU (described under publications). Team members presented widely at scientific conferences and as part of invited seminars both nationally and internationally. Short courses related to the work on the project were also presented by team members Dorea and Rosa, which further enhance the connectivity of our group to others working in or entering this area of research.

Publications

  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Siegford JM. 2023. Does automated behavioral monitoring inevitably lead to improved pig welfare? Journal of Animal Science. 101(Supplement S3): 317-318.
  • Type: Journal Articles Status: Published Year Published: 2024 Citation: Siegford JM, Steibel JP, Han J, Benjamin M, Brown-Brandl T, D�rea JRR, Morris D, Norton T, Psota E, Rosa GJM. 2023. The quest to develop automated systems for monitoring animal behavior. Applied Animal Behaviour Science 265:106000. doi:10.1016/j.applanim.2023.106000.
  • Type: Other Status: Published Year Published: 2024 Citation: Shumaly M, Steibel JP. 230 Using phenotypes with uncertain identification leveraging computer vision models. Journal of Animal Science. 2024 May 1;102(Supplement_2):25-6.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Steibel, J. 2023. Use of Hardware and Sensors Towards Phenomics to Deliver Complex Data and Advance Animal Breeding. Presented at: Beef Improvement Federation. (Oral Presentation)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Han J, Dorea JR, Norton T, Parmiggiani A, Morris D, Siegford J, Steibel JP. 2023. Publicly available datasets for computer vision in precision livestock farming: A review. US Precision Livestock Farming 2023: Conference Proceedings of the 2nd US Precision Livestock Farming Conference, Knoxville, TN, May 21-24, 2023. 2:618-625
  • Type: Websites Status: Published Year Published: 2024 Citation: Computer Vision in Precision Livestock Farming Webinar Series. Updated 2024. https://www.canr.msu.edu/precision-agriculture/Precision-Livestock-Farming-Webinar-Series/
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Paudel S, Brown-Brandl T, Rohrer G, Sharma SR. 2023. Individual pigs identification using deep learning. In 2023 ASABE Annual International Meeting (p. 1). American Society of Agricultural and Biological Engineers.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Rosa, G. J. M. Integration of Environomics and Genomics. 46th American Dairy Science Association (ADSA) Discover Conference, Itasca, Illinois. May 6-9, 2024.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2024 Citation: Rosa, G. J. M. Combining Big Data Analytics and Omics Techniques to Improve Beef Cattle Selection and Production. XXXI Plant & Animal Genomes (PAG) Conference, San Diego, CA. January 12-17, 2024.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Rosa, G. J. M. The Use of AI and Statistics in Optimising the Use of Livestock Big Data. Conference on Artificial Intelligence and Data Analytics for Improved Veterinary Care. University of Surrey, Guildford, UK. December 11, 2023. (online)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Rosa, G. J. M., Lourenco, D., Rowan, T. N., Brito, L. F., Gondro, C., Huang, J. and Valle de Souza, S. Integrating Enviromics, Genomics, and Machine Learning for Precision Breeding of Resilient Livestock. 2023 ASAS-CSAS-WSASAS Annual Meeting. Albuquerque, NM. July 16-20, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Rosa, G. J. M., Dorea, J. R. R. and Hernandez, L. Digital Technologies and Machine Learning: A New Way to Look at Novel Traits at Spatial and Temporal Dimensions. 2023 ASAS-CSAS-WSASAS Annual Meeting. Albuquerque, NM. July 16-20, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Rosa, G. J. M., Hernandez, L and Dorea, J. R. R. Digital Technologies and Machine Learning: A New Way to Look at Novel Traits at Spatial and Temporal Dimensions. 25th Congress of the Italian Animal Science and Production Association (ASPA), Bari - Italy, June 13-16, 2023.
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Freitas, L. A., Ferreira, R. E. P., Savegnago, R. P., Dorea, J. R. R., Stafuzza, N. B., Rosa, G. J. M. and Paz, C. C. P. Image analysis to automatically classify anemia based on Famacha� score in sheep using ocular conjunctiva images. Translational Animal Science 7: txad118, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Cominotte, A., Fernandes, A., D�rea, J., Rosa, G. J. M., Torres, R., Pereira, G., Baldassini, W. and Neto, O. M. Use of biometric images to predict body weight and hot carcassweight of Nellore cattle. Animals 13: 1679, 2023.
  • Type: Other Status: Other Year Published: 2024 Citation: Rosa, G. J. M. Digital Technologies and Machine Learning: A New Way to Look at Novel Traits at Spatial and Temporal Dimensions, Invited Seminar at Chungnam National University (CNU), Daejeon  South Korea. February 7th, 2024.
  • Type: Other Status: Other Year Published: 2023 Citation: Rosa, G. J. M. Genomics and Phenomics Applied to Aquaculture Breeding. Invited Seminar (virtual) at University of Chile, Santiago, Chile. November 29th, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Steibel JP, Brown-Brandl T, Rosa GJM, Siegford JM, Psota E, Benjamin M, Morris D, Dorea JRR, Norton T. 2023. Progress report on the coordinated innovation network for advancing computer vision in precision livestock farming. In: US Precision Livestock Farming 2023: Conference Proceedings of the 2nd US Precision Livestock Farming Conference, Knoxville, TN, May 21-24, 2023. 2:146-150.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Ferreira, R. E. P., T. Bresolin, J. C. F. Silva and J. R. R. Dorea. 2023. Democratizing the access to artificial intelligence solutions for underrepresented and non-expert communities. In: US Precision Livestock Farming 2023: Conference Proceedings of the 2nd US Precision Livestock Farming Conference, Knoxville, TN, May 21-24, 2023. 2:43-49.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Bresolin, T., R. E. P. Ferreira, G. J. M. Rosa, and J. R. R. Dorea. 2023. Computer vision on the edge: A computing framework for high-throughput phenotyping in livestock operations. In: US Precision Livestock Farming 2023: Conference Proceedings of the 2nd US Precision Livestock Farming Conference, Knoxville, TN, May 21-24, 2023. 2:151-156.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Ferreira, R. E. P., M. C. Ferris, and J. R. R. Dorea. 2023. Optimizing training sets for individual identification of dairy cows. J. Dairy Sci. 106:429. Annual Meeting ADSA, Ottawa, Canada.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Dorea JRR. 2023. Invited presentation: Transforming Dairy Farm Management with the Power of Artificial Intelligence. IDF World Dairy Summit. Chicago-IL, October 15th, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Dorea JRR. 2023. Precision Livestock Farming for Optimal Management. SIAM, Moroccan International Agricultural Show. Meknes, Morocco. May 4th, 2023.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2023 Citation: Dorea JRR. 2023. Leveraging Artificial Intelligence to Optimize Farm Management Decisions. National Agricultural Producers Data Cooperative. University of Nebraska-Lincoln. September 19th, 2023.


Progress 03/01/22 to 02/28/23

Outputs
Target Audience: Animal Scientists, Animal Breeders, Engineers, data analysts and computer scientists interested in the application of computer vision in animal production systems. Changes/Problems:A main challenge we faced in the past year was that Dr. Juan Steibel (the initial PD) accepted a position in the department of Animal Science at Iowa State University. The decision was made to keep the project at MSU and Dr. Siegford is acting now as overall project PD, and Dr. Steibel has remained on the project as a co-PD. No funds were requested to be transferred to ISU, but instead as part of his start up Dr. Steibel procured funds to support his activities related to this project. One consequence of this was that parts of the planed activities for the reporting year were delayed. For instance: the first challenge was not released and some webinars editions were missed. But the team is back on track to accomplish the proposed work: two graduate students have been recruited at Iowa State University, and UW Madison and UNL are generating data for analytic challenges in the coming year. What opportunities for training and professional development has the project provided?Undergraduate and graduate students have been involved in the project in the labs of several of the project PIs. These students have received training in and opportunities to engage in coding and data analysis projects related to precision livestock farming. PhD student Junjie Han graduated in fall 2022 and was the lead author on 2 journal articles published this year from the grant as well as on a forthcoming presentation and preceedings paper and the 2nd US PLF Conference. As a result of his work on projects with PD Steibel, he has had opportunities to collaborate and co-author with PD Norton and one of his postdocs (who has since returned to China). Two current PhD students are engaged in work with PD Morris and PD Brown-Brandl. Additionally, the webinars (and resulting recordings which are available online) have been attended (and viewed) by graduate students and postdocs within and beyond the team, providing them with opportunities to network as well as exposure to a range of problems and approaches in computer vision to solving PLF-related problems. How have the results been disseminated to communities of interest?The webinar series has been used to disseminate part of the project's results (e.g., talks by coPDs Steibel, Psota, and Dorea). Invited presentations have been delivered by PDs as reported. Papers in reputable peer-reviewed scientific journals were also published. Several invited presentations and contributed presentations were delivered in several venues, including the following scientific meetings: American Dairy Science Association, European Commission Precision Livestock Farming, Argentinian Society of Animal Science, World Congress of Genetics Applied to Livestock, International Society for Applied Ethology, Conference on Statistics Applied to Agriculture and Natural Resources. What do you plan to do during the next reporting period to accomplish the goals?1) We will keep broadening the webinar series to include no-coPI presenters and especially presenters contacted through activities of this grant (webinar participants, challenge participants, etc) in order to grow the CIN and document its impact. 2) The team will add at least two new curated datasets to the public repository. 3) We plan to deliver at least one analytical challenge around the available data collected and summarized during the first reporting period. 4) We will enerate novel data for challenges in successive years.

Impacts
What was accomplished under these goals? A review of publicly available data for application of computer vision in animal production systems (Objectives 1a and 1b) was finalized and submitted as a short paper for the USPLF congress and it will be presented in the meeting in May 2023. 20 datasets and their corresponding published papers were identified, and they are being summarized for publication. Last year an OSF repository was established (see products, dataset repository) to create a permanent repository of datasets related to this project, and a first dataset was added to that repository (Animal Activty). The full analysis of that Animal Activity dataset was completed and published in this reporting year results were widely disseminated through invited presentations. This year we added a second dataset to the repository on Pseudolabeling for Animal ID, and the baseline analysis of this dataset was finished and the paper is under review. A webinar series on computer vision in PLF (Objective 2b) was started in the previous reporting year and continued through this year. In the current reporting period 5 webinars were presented. The initial webinars were presented by the core group of the CIN, but after that, other groups were invited to present. This series caters to a broad and diverse audience from every continent and from several disciplines related to computer vision, genetics, engineering and animal production. The feedback from the audience was been excellent and when our group reports/presents results in conferences, we are receiving now requests of other groups to schedule presentations in the webinar series.

Publications

  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Steibel, J.P. Validation of computer vision algorithms for behavioural phenotyping of pigs. World Congress of Genetics Applied to Livestock. Rotterdam, The Netherlands. (Oral Presentation)
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Han, J., Siegford, J., Colbry, D., Lesiyon, R., Bosgraaf, A., Chen, C., Tomas Norton & Steibel, J. P. (2023). Evaluation of computer vision for detecting agonistic behavior of pigs in a single-space feeding stall through blocked cross-validation strategies. Computers and Electronics in Agriculture, 204, 107520.
  • Type: Journal Articles Status: Published Year Published: 2023 Citation: Han, J., Siegford, J., de los Campos, G., Tempelman, R. J., Gondro, C., & Steibel, J. P. (2022). Analysis of social interactions in group-housed animals using dyadic linear models. Applied Animal Behaviour Science, 256, 105747.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Steibel, J.P. Computer vision and machine learning for phenotyping. ARPAS Symposium: Artificial Intelligence and Machine Learning in Dairy Production Systems. ADSA. (Invited Oral Presentation)
  • Type: Conference Papers and Presentations Status: Other Year Published: 2022 Citation: Steibel, J.P. Remote visualization of animal interactions: automatic detection of social behavior and its application to swine genetics and management. IV Workshop em Estudos Avan�ados em Precis�o Animal. Unicamp, Brazil. Virtual Presentation. (Oral Presentation)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Steibel, J.P. What are animal scientists learning from using deep learning? Conference on Applied Statistics in Agriculture and Natural Resources. Logan, Utah. (Oral Presentation)
  • Type: Conference Papers and Presentations Status: Other Year Published: 2022 Citation: Steibel, J.P. Data Science challenges in the livestock phenomics-genomics era. NRSP-8 Annual Meeting. San Diego, California. (Oral Presentation)
  • Type: Conference Papers and Presentations Status: Other Year Published: 2022 Citation: Steibel, J.P. Genomic approaches to selecting animals that will thrive in group-housed settings. Topigs Norsvin's Annual Health & Welfare Meeting. Virtual presentation. (Invited Oral Presentation)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Siegford J*, Steibel J, Han J, Benjamin M, Brown-Brandl T, Dorea JRR, Morris D, Norton T, Psota E, Rosa GJ. 2022. The quest to develop automated systems for monitoring animal behaviour. Proceedings of the 55th Congress of the International Society for Applied Ethology. 55:3. (Oral Plenary Presentation)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Steibel J.P., Brown-Brandl T., Rosa, G.J.M., Siegford, J.M., Psota E., Benjamin M., Morris D., Dorea, J., Norton T. (2022). Progress report on the Coordinated Innovation Network for Advancing Computer Vision in Precision Livestock Farming. Proceedings of the 10th European Conference on Precision Livestock Farming. (Oral presentation)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Steibel J.P., J. Han, C. Chen, J. Siegford, T. Norton, D. (2022) Validation of computer vision algorithms for classifying video segments applied to behavioural phenotyping of pigs. Proceedings of the 12th World Congress on Genetics Applied to Livestock Production.
  • Type: Conference Papers and Presentations Status: Other Year Published: 2022 Citation: Siegford, J.M. Animal behavior and welfare: what can technology tell us? IMAGEN & Breed4Food Individual Tracking symposium, Wageningen, The Netherlands. (Invited Oral Presentation.)
  • Type: Conference Papers and Presentations Status: Other Year Published: 2023 Citation: Morris D, Siegford J, Grebey T. (2023) Floor Egg Detection for Cage Free Hens, AI for Ag Conference, Orlando, FL. (Oral Presentation)
  • Type: Conference Papers and Presentations Status: Published Year Published: 2022 Citation: Bresolin, T., R. E. P. Ferreira, J. R. R. Dorea. 2022. Effect of Camera Exposure Time on Image Segmentation and Body Weight Prediction. Journal of Animal Science https://doi.org/10.1093/jas/skac247.586. (Abstract)
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Ferreira, R. E. P., T. Bresolin, G. J. M. Rosa, J. R. R. Dorea. 2022. Using dorsal surface for individual identification of dairy calves through 3D deep learning algorithms. Computer and Electronics in Agriculture. 201:107272.
  • Type: Journal Articles Status: Published Year Published: 2022 Citation: Bresolin, T., R. E. P. Ferreira, F. Reyes, J. Van Os, J. R. R. Dorea. 2022. Feeding behavior of dairy heifers monitored through computer vision systems. Journal of Dairy Science. 106 (1), 664-675.


Progress 03/01/21 to 02/28/22

Outputs
Target Audience:Animal Scientists, Animal Breeders, Engineers, data analysts and computer scientistsinterested in the application of computer vision in animal production systems. Changes/Problems:A challenge in 2021 was the recruitment of new students at our institutions. Due to restrictions imposed by the pandemic, we did not have access to as a diverse applicant pool as usual (international graduate student applications were at a low time low). Zoom fatigue was really noticeable among coPI and potential participants and presenters in our webinar series. We delayed the start of the webinar series and concentrated more on data collection and annotation. We are now on track with the planned activities and webinar participation is at an all time high. For 2022 we broaden the base from which we are recruiting students (several graduate programs across MSU and UNL). We have students starting in Jan 2022 and in August 2022. What opportunities for training and professional development has the project provided?Two graduate students (One at MSU and one at KUL) and two undergraduate students (MSU) have been training as part of this grant. How have the results been disseminated to communities of interest?The webinar Series has been used to diseminate part of the results (e.g: talks by coPIs Steibel, Psota and Dorea). Invited presentations have been delivered as reported. Papers were also published. What do you plan to do during the next reporting period to accomplish the goals?1) Broaden the webinar series to include no-coPI presenters and especially presenters contacted through activities of this grant (webinar participants, challenge participants, etc) in order to grow the CIN and document its impact. 2) Publish and maintain a repository of available datasets for computer vision. 3) Deliver at least one analytical challenge around the available data collected and summarized during the first reporting period. 4) generate novel data for challenges in successive years.

Impacts
What was accomplished under these goals? A review of publicly available data for application of computer vision in animal production systems (objectives 1a and 1b) is underway. 21 datasets and their corresponding published papers were identified and they are being summarized for publication. An OSF website has been established to create a permanent reposity and datasets are being added to this resources (Or linked to if already in an OSF repository). A dataset with recording and annotations of pig interactionsat the feeder (Objective 1b) has been collected. This dataset includes over 21,000 30-frame long video episodes annotated with the type of behavioral interaction. A baseline analysis has been completed, which includes alternative validation schemes. The corresponding paper is about to be submitted for review and it will constitute the basis for an analytical challenge (objective 1a) in 2022. A webinar series on computer vision in PLF (Objective 2b) was started. This series catters to a broad and diverse audience from every continent and from several disciplines related to computer vision, genetics, engineering and animal production.

Publications

  • Type: Journal Articles Status: Published Year Published: 2021 Citation: P�rez-Enciso, M., & Steibel, J. P. (2021). Phenomes: the current frontier in animal breeding. Genetics Selection Evolution, 53(1), 1-10.
  • Type: Journal Articles Status: Published Year Published: 2021 Citation: Han, Junjie, Cedric Gondro, Kenneth Reid, and Juan P. Steibel. "Heuristic hyperparameter optimization of deep learning models for genomic prediction." G3 11, no. 7 (2021): jkab032.
  • Type: Conference Papers and Presentations Status: Published Year Published: 2021 Citation: Steibel, Juan P. Phenotyping for Precision Swine Management and Breeding. (2021). Joint National Swine Improvement Federation and Poultry Breeder's Roundtable annual meeting. Saint Louis, MI