Source: EMERY ANIMAL HEALTH submitted to
IMPROVING HERD REPRODUCTIVE EFFICIENCY UTILIZING ARTIFICIAL INTELLIGENCE ASSISTED SPERM ANALYSIS & SUPPORT APPS
Sponsoring Institution
National Institute of Food and Agriculture
Project Status
COMPLETE
Funding Source
SMALL BUSINESS GRANT
Reporting Frequency
Annual
Accession No.
1025817
Grant No.
2021-33530-34400
Cumulative Award Amt.
$100,000.00
Proposal No.
2021-00998
Multistate No.
(N/A)
Project Start Date
Jul 1, 2021
Project End Date
Feb 28, 2023
Grant Year
2021
Program Code
[8.3]- Animal Production & Protection
Project Director
Jensen, G.
Recipient Organization
EMERY ANIMAL HEALTH
470 W HWY 29
CASTLE DALE,UT 84513
Performing Department
(N/A)
Non Technical Summary
Bulls are a major contributing factor to the reproductive efficiency and economic viability of beef and dairy businesses. However, current speriograms are incomplete, subjective,unreliable, and without a system for bull buyers to easily access and understand results. Sperm morphology is a critical component of fertility evaluation during a breeding soundness examination and freezing semen. Using machine learning, a form of artificial intelligence, we will develop a chute side semen analysis tool that will compile an objective complete differential spermiogram with consistent accuracy unmatched by humans. This chute side diagnostic tool will remove the subjective nature of semen analysis. Once completed will be an invaluable tool to quickly and accurately assess semen during a bull breeding soundness exam. Accompanying apps will help veterinarians and cow-calf producers understand bull testing results needed tobetter make management decisions for improving herd fertility. Accurate consistent results will be assured through defined processes, careful selection of microscopic equipment, reliable image data, and deep machine learning technologies.Veterinarians, artificial insemination centers, researchers, and otherscan use this chute side semen evaluation toolchute sidefor quick accurateresults. Cow-calf producers will improveprofits through improvingreproductive efficiency by using bulls and semen with increasedpotential for higher pregnancy rates. The US beef industry can realize up to $1.2 billion from improved production for improved sustainability.
Animal Health Component
0%
Research Effort Categories
Basic
0%
Applied
0%
Developmental
100%
Classification

Knowledge Area (KA)Subject of Investigation (SOI)Field of Science (FOS)Percent
30133991081100%
Knowledge Area
301 - Reproductive Performance of Animals;

Subject Of Investigation
3399 - Beef cattle, general/other;

Field Of Science
1081 - Breeding;
Goals / Objectives
Goals:Our final goal is to createchute side diagnostic tool that will objectively analyze bull semen for motility and morphology. This project phase will focus on two areas: 1) Obtaining a large data set of quality labeled sperm images. 2) Determining the best machine learning (ML) algorithms or sets of algorithms for optimal results.At completion we will have a large dataset of bull sperm labeled images, know where our data set weakness may be and what we need both in additional quantity of images and potential quality of labels and image quality for bull sperm morphological assessment.Have a functional set of processes and ML algorithms that will create reliable results for bull sperm morphological analysis.Technical ObjectivesCreate a high-quality data set with 20,000 - 30,000 labeled images of sperm normal and abnormal morphology for ML training.Identify the most promising open-source CNN architectures and use these to train a model, or multiple models, for the purposes of comprehensive bull sperm morphological analysis.Develop standardized data processing flow for uploading images from the microscope camera into the application.Develop a beta, proof of concept, application that can deploy the model developed in objective #2 and obtain prediction results in a reasonable amount of time on a common mid-level laptop or mobile device.
Project Methods
DatasetsTo compile an effective data set with the ability to create modules which will identify specific sperm abnormalities with a high percent of recall and precision we will start with a list and example images of the labels (morphological abnormalities) to use for labeling. This reference data set will represent the variations of morphological labels and may be adjusted as needed. The reference data set will be reviewed by a member of the Collage of Theriogenology.We will create labeled datasets to be used in Google's computer visions training. Our final data sets will have at least 1,000 samples of each label. The ML training creates a module, and our tested results will give us the recall and precision of the model created from the data set. The data set will then be reviewed to identify errors or needs which can improve recall and precision. These include changes like breaking down morphological labels into more distinct morphological characteristics or excluding images of sperm that are out of focus or being obscured by other objects.The refined datasets will then be used to compare different labeling techniques. Bonding boxes are used in labeling Google's computer vision ML, pixel labeling technics are used in APEER, Zeiss's annotation methods, which takes more time but may be a superior way to segment individual sperm cells from each other and other types of material within the microscopic field. We will consult with individuals who have experience in this area and run A/B tests where needed to compare methods. This can be accomplished by taking the exact same data set, labeling sperm using the different methods and then compare results.Machine learning architype and processes.Application development will utilize common code languages such as Python and Java Script, as well as open-source frameworks for ML model development and deployment, including the popular and widely used TensorFlow and Keras frameworks. We will also make use of open-source CNN architectures available on conde sharing platforms such as GitHub as we evaluate various network architectures for the specific problem of sperm morphology detection for practical usage in the field. We currently have sufficient images and labeled datasets to begin this process. Therefore, app development and dataset creation can occur concurrently.We will focus first on developing a model that can accurately identify readable sperm and tally the total sperm count in an image. We will then develop a model that will identify morphological abnormalities present in the individually identified sperm. We will be making use of the approach popularized by Andrew Ng in which we build the first model quickly and then iterate to identify the most successful hyperparameters and data processing strategies give the characteristics of the unique problem domain. We will start with a basic set of abnormalities and then iterate through this process as many times as needed. To extend the number of abnormalities that can be comprehensively predicted.In evaluating the resulting trained model(s) we will be looking for cross validation accuracy, recall, and precision, as well as the feasibility of using the model in an application. This includes analyzing the physical storage size of the saved model, including parameters and weights, its impact on application footprint, and model performance when running prediction locally on hardware not specialized for machine learning computation.We will run A/B trials to compare two ML architectures. The results can be compared to select the superior method. This can be repeated by making changes to the ML methods and architypes each time comparing results to select the superior process and achieve a high percent recall and precision of morphological labels.Once reasonable confidence in model architecture and feasibility has been achieved, we will begin laying the groundwork for the basic application, iterating, and adding features as time allows, including but not limited to:Identifying and building skeletal application code targeted to specific devices, starting with environments most likely to be commonly used by veterinary consumers of the application (i.e. Android, iOS, Windows/.NET)Setting up cloud-based application serving and centralized data aggregationSyncing prediction results obtained in offline mode to a centralized database.Increasing the quality of BBSE data gathered by the application.Increase/improve analysis generated from gathered data and insights from that data.Diversification of users to include client user views in addition to veterinary user views.A write-up or record of each iteration will be created including algorithms and changes made including results of each. We will also create a tabular record for easy comparison and analysis of results. These will be reviewed by the PI and AI engineer on a regular basis to make selections and changes as needed base on the results.

Progress 07/01/21 to 02/28/23

Outputs
Target Audience:The target audience is veterinarians, veterinary academia, bull seedstock producers, cattle producers. Changes/Problems:I put together a "Final Report" that is a 25 page report. Is "Final Reprot" use in more than one area. It gets confusing when the same terms are used. What opportunities for training and professional development has the project provided?Opportunities for professional development The profession of veterinary medicine has a board certification in animal reproduction. Certification allows professionals to be part of the "College of Theriogenologists." The Society for Theriogenology has its roots in bull breeding soundness exams and has been responsible for creating the guidelines used by veterinarians to perform these services. Struggles with the human factors causing inaccuracies has plagued bull breeding soundness exams for decades and has been a source of frustration for many. Veterinarians and veterinary specialists do not have knowledge and training in these human imperfections and handling of the types of conflicts of interest and biases, such as the "Self-serving bias," that creates these frustrations. The incentives for bull seed stock producers currently do not align with the values needed for successful industry wide improvements to use bulls with a higher potential reproductive influence. Seedstock producers do not have an incentive to improve management or selection pressures to improve bulls, rather they select evaluators who will give them the results they feel they need. This increases the chances of less fertile bulls being used and going unseen by the bull owners. Our early commercialization research through multiple interviews with veterinarians, seedstock producers, and the end users (bull buyers) has helped us to better understand the conflicts of interest inherent in the BBSE systems. We have also learned that some veterinarians can experience moral injury which can contribute to increased mental health concerns, suicidality, and the increased potential of leaving rural practice or the profession. The pressures and needs to keep great client relationships and maintain or improve the quality of veterinary services creates cognitive dissonance, a form of mental anguish. Bull seedstock producers often truly believe that close to 100% of their bulls should "pass" the exam, if they don't it is not the bull's fault, the producers reason; "If it was a problem, then why am I not getting more complaints." There are methods to help address these human concerns. The methods needed are found in the business processes of understanding the problem, identifying possible solutions, and then testing these hypotheses while bringing products such as the BBSE to market. We formulate processes and methods that use scientific knowledge about bull and herd fertility that customers can also use in ways that solve or mitigate the problems described above. Lean startup strategies are one such method. The field of psychology also has specialized areas that help us to better understand these human aspects. We must incorporate these other fields of study to create successful programs. Veterinary specialists can seek out help from other professionals in areas outside of our own industries. Too often it becomes our own nature to believe we have the intelligence / knowledge and skill to do it all on our own, the overconfidence bias. Our veterinary community can improve by being more open to listening for understanding, asking questions to improve understanding and fighting the urge to think we a right. We do not know the things we do not know and the only way to discover these areas is to always be curious and willing to be wrong. There are veterinarians suffering quietly in veterinary practice, surveys often can not bring out the understanding needed, we can listen with compassion and humility, and a desire to truly learn from each other to improve our profession. How have the results been disseminated to communities of interest?As a member of the AABP reproductive committee I have tried to bring up the concerns and problems found with in bull breeding soundness exams. The research and objectives of the project are to better understand how to design and use artificial intellagence as a tool to remorve the human variabilities and subjetivities. However, we also discoved that this on its own will not be enough. Unforturnatly, dialogue within the professional organization has be very difficult and little progress has been made. What do you plan to do during the next reporting period to accomplish the goals?We are hoping to win the phase II grant. Part of this phase will also help to address the conflict of interest and search to solve this problem. We can then work on publishing these findings in hopes to create more positive change for the veterinary and cattle industries.

Impacts
What was accomplished under these goals? Purpose The purpose of this project is to address the issue of low reproductive efficiency in cattle and the stagnation of improvement in this area over the last few decades seen in the US. Veterinary bull breeding soundness exams (BBSE) are a proven tool to help select or cull less fertile bulls. A vital portion of the BBSE is sperm morphology which is highly correlated with calf output and time to conception. The subjectivity and human imperfections cause variations of results between practitioners. There is a lack of standardization, no certification programs, and no quality control. These problems make any meaningful interpretation or comparisons of results useless. Furthermore, seedstock bull producers are incentivized to select veterinarians who will "pass" a higher percentage of bulls allowing them to sell more bulls and increase profitability, thus penalizing veterinarians who wish to improve the quality of the BBSEs they offer, where doing so identifies a higher percentage of subfertile bulls and decreases the seed stock producer's income. Using Artificial Intelligence (AI), specifically computer vision (CV), to identify sperm morphological abnormalities is a feasible method to remove the subjectivity of the human evaluation. However, it will not remove the conflict of interest that is inherent in today's BBSE systems. We successfully competed in each of our four objectives and showed that CV is a feasible method to identify sperm abnormal structures. Objective #1: Dataset creation and annotations 20,818 usable images were obtained and used in various datasets for the purpose of AI training. These images contained 43,205 annotations representing several sperm morphological abnormalities. The datasets underwent several iterations to improve annotation practices and handle different sets of abnormalities separately. Preprocessing and image augmentations were applied to enhance the datasets and address challenges related to the specific sets. Objective #2: CNN Architecture Selection and Model Development YOLOv5 CNN architecture was chosen as a viable option for assessing bull sperm morphology. Using the created datasets and YOLOv5, models were trained and achieved acceptable results in detection and classifying various abnormalities. However, challenges in small object detection were identified. These challenges will be addressed in Phase II through additional preprocessing, augmentation techniques, and increasing the dataset size. Objective #3: Data Processing Flow We successfully developed methods to capture images using four different types of microscope cameras and uploaded the images both to the cloud and on an edge device using a prototype application. This has created a standard flow for the analysis. Objective #4: Model Deployment The images obtained from the microscope camera were processed using the developed models in Objective #2. Inference was successfully performed on the images, demonstrating the ability to deploy the trained models on captured microscope images. Conclusion Phase I of the project demonstrated the feasibility of utilizing AI assisted sperm analysis to improve bull breeding soundness examinations. Creating a non-subjective analysis that is both effective and efficient will improve veterinarian's ability to assess the potential of bull fertility and create more integrity of the BBSE system. The conflict of interest will also need to be addressed to achieve real meaningful progress in helping to improve reproductive efficiencies of cattle herds.

Publications


    Progress 07/01/21 to 06/30/22

    Outputs
    Target Audience: Nothing Reported Changes/Problems: Nothing Reported What opportunities for training and professional development has the project provided? Nothing Reported How have the results been disseminated to communities of interest? Nothing Reported What do you plan to do during the next reporting period to accomplish the goals?Begin work on the problems of commercialization by interviewing veterinarians, bull seed stock producers, and bull buyers.

    Impacts
    What was accomplished under these goals? Two achieve the goals of creating a chute side diagnostic tool for an objective bull semen analysis we have made significant progress on each of the four objectives. We are using Roboflow to create image datasets, annotate the images, and run training using YOLOv5. One of the advantages of Roblflow is having a data scientist available as a consultant on the artificial intellagence and computer visionobjectives. We have created a manual with example images of sperm morphological abnormalities. This is used by annotators as a referance for correct taxonomy of sperm morphology. We have found that some morphological structures with one name need to be divided for better CV understanding of the mechine learning results. A list of our taxonomy has be created and is modified as needed. We extended the grant time frame to better understand the problems around bull breeding soundness exams (BBSE) in veterinary practice. We have become concerned that the creation of a quality chute side anylitical tool on its own may not be suffiecient to solve many of the problems we believe are inherint in todays BBSE system. Veterinarians lose clients, specifically bull seedstock producers, when they create a higher quality semen analysis. This occurs becuse by doing so we identify more sperm having abnormalities and thus fewer bulls "pass" the evauations.

    Publications