Progress 06/15/23 to 06/14/24
Outputs
Target Audience: The BrAPI project strives to assist anyone involved with the standardization, transportation, and management of breeding data and related metadata. This includes PI's, breeders, biologists, computer scientists, data scientists, and other experts, working in organizations around the world. The current BrAPI community has about 250 members representing over 37 different organizations involved with breeding research and data management. Changes/Problems:RegardingObjective 1, as discussed in the previous annual report, releasing a new version of the specification every year was found to be unfeasible and undesired by the community. Work has begun on V2.2 which will address the highest priority community needs from the past two years. RegardingObjective 2, the first two tasks are complete as planned. The second two tasks regarding documentation for "Web Sockets" and "Message Queues" will likely get replaced by the need for RDF and NoSQL documentation. These are somewhat newer technologies and will fit the needs of the community much better. RegardingObjective 3, the task "Build an Interactive BrAPI Tutorial" is behind schedule as it was deemed lower priority than some of the other tasks inObjective 3which have proceeded on time as planned. RegardingObjective 4, there was a call for sabbatical project proposals last year, however no one in the community took advantage of the opportunity. The travel funds for the sabbaticals were ultimately reallocated to support travel to the Wittenberg hackathon for those who would not have been able to attend otherwise. Objective 5is proceeding on schedule as planned, with no changes. What opportunities for training and professional development has the project provided?The community hackathons discussed inObjective 4have created opportunities for training and professional development for members of the BrAPI community. Each community hackathon begins with a formal "BrAPI 101" training class to bring everyone up to speed on what BrAPI is, how it works, and how to get started. This class can be requested for specific groups at any time throughout the year, but it will always be presented at least twice a year publicly for the hackathons. This year, there were two additional classes introduced during the hackathons. "BrAPI Domain Model Review" extended the content in the "BrAPI 101" class for more veteran members who wanted to explore a deeper knowledge of the specification. The "Security and Authentication" discussion taught attendees how to use the OAuth2 infrastructure in the context of BrAPI, as well as some general web security best practices. How have the results been disseminated to communities of interest?All news, materials, events, tools and updates in the BrAPI project are communicated to the BrAPI community through the community mailing list, the community Slack workspace, and posted on the project website (brapi.org). The community hackathons are also used as a forum to directly discuss new ideas, gather community feedback, and present important project results. Sometimes, large or important results are communicated to other relevant communities through conference and webinar presentations. What do you plan to do during the next reporting period to accomplish the goals? Develop and release BrAPI v2.2 Enhance the BrAPI Test Server into a production ready server which can be adopted by the community Develop best practice documentation and standards for the BrAPI specification Create a set of self-guided, interactive, training material Organize two hackathons Conduct another round of the BrAPI Sabbatical program Continue attending and presenting at relevant conferences
Impacts
What was accomplished under these goals?
Objective 1: Expand the functionality in the RESTful API standard to fully support modern breeding applications. The BrAPI project has established a cycle continuous enhancement and improvement based on the needs of the community. The BrAPI specification version V2.1 was officially released in July of 2022, and since then community groups have been implementing the new standard and sharing feedback. That feedback is collected and tracked through the GitHub issues page, available for public review and commentary. This year, it was determined by the project leadership and advisory board that there are enough issues and enhancement requests to justify the creation of the next version of the BrAPI specification. The development of BrAPI V2.2 officially began in August 2024, but there has been a lot of work done over the past year in preparation for the technical changes. The BrAPI Project Coordinator worked directly with community individuals and teams to discuss each issue. The issues are reviewed, discussed, and expanded on to make sure requirements are properly gathered and documented. One of the main considerations for each change request is to determine if it is specific to a particular team or if it can be generalized and useful to the wider community. This often requires additional discussion with community representatives, gaining general support for an idea during community meetings and hackathons. Objective 2: Implement additional API technologies to achieve optimal performance for a wide range of use cases. During the past year, the BrAPI data model schema has been successfully extracted from the original RESTful documentation. Two community groups were able to successfully use this extracted schema to implement GraphQL API servers. This has been a major goal for the past two years, allowing the BrAPI spec to be used in multiple different types of technologies. Based on this work, the upcoming V2.2 of the spec will be built as a generic schema from the beginning. Then automated tools will be used to create the RESTful documentation, GraphQL documentation, RDF documentation, and any other technologies requested by the community. This change will better support the diversity of the community as well as better future-proof the specification. There are still ongoing discussions in the community about how these different technologies might interact with each other. There is a general consensus to keep REST as the default technology used by most groups, but that may change as different technologies gain favor. Objective 3: Maintain and expand the project infrastructure, supporting software developers to build implementations of the standard quickly and efficiently. Over the past year, most of the existing project infrastructure has remained stable and has required only occasional maintenance. The primary expansion to the infrastructure has been the addition of the automated tools mentioned inObjective 2to manage the BrAPI data model schema. A tool has been built to validate the structure of the schema, making sure it conforms to the JSON Schema protocol, as well as some community best practices specific to the BrAPI project. Development is ongoing for a conversion tool which will be used to take the base schema documents and produce official API documentation (RESTful, GraphQL, RDF, etc). Objective 4: Provide opportunities for direct collaboration between groups, and provide community-building activities. As planned, two BrAPI community hackathons were executed this year. The first, on October 23-27, 2023, was completely virtual. There were 41 participants representing 23 different organizations from around the world, and approximately one fourth of the attendees were new to the BrAPI community. The second hackathon, on April 15-19, 2024, was an in-person event in Wittenberg, Germany, hosted by IPK Gatersleben and FZ Juelich. There were 31 participants representing 19 different organizations from around the world, and more than one third of the attendees were new to the BrAPI community. The biggest collaboration this year was the writing of a new BrAPI Community manuscript. This new paper is intended to be less technical than the first BrAPI paper, with a focus on the different BrAPI compatible tools available. 33 software projects are presented with a short description and an explanation of how BrAPI enhances the tool. This paper was a large collaborative writing project with 76 contributing authors, organized by the BrAPI Project Coordinator. It has been submitted to PLOS Computational Biology, awaiting review. Objective 5: Expand the BrAPI community, with a focus on continued diversity and representation of new domains. Along with the community hackathons, discussed inObjective 4, conferences and webinars are a great way to introduce new people to BrAPI and grow the community. In the past year, the BrAPI project coordinator has presented BrAPI at the following events: the BTI Breedbase Symposium 2023, the Breeding Insight OnRamp Seminar Series, the Cornell Plant Breeding & Genetics Seminar Series, the Plant & Animal Genome Conference 2024, and the NIFA PD Meeting. Each of these events targeted a different community within the domain of international agriculture and breeding. Each event led to follow-up conversations from at least one attendee/group. The company BASF officially joined the BrAPI Community this year. BASF is a major player in private sector breeding and seed production. This new relationship was made possible by some of the work towards a GraphQL standard mentioned inObjective 2. This represents a significant step in one of the BrAPI projects major goals to gain support from more private sector companies.
Publications
- Type:
Conference Papers and Presentations
Status:
Published
Year Published:
2024
Citation:
Peter Selby, Applications and Impacts of the BrAPI Project on Plant Breeding, PAG31, January 13, 2024. https://pag.confex.com/pag/31/meetingapp.cgi/Paper/54119
|
Progress 06/15/22 to 06/14/23
Outputs
Target Audience:The BrAPI project strives to assist anyone involved with the standardization, transportation, and management of breeding data and related metadata. This includes PI's, breeders, biologists, computer scientists, data scientists, and other experts, working in organizations around the world. The current BrAPI community has about 250 members representing over 37 different organizations involved with breeding research and data management. Changes/Problems:In Goal1, the original plan was to release a new version of the specification every year. After polling the community and gathering some anecdotal feedback during the hackathon, the project leadership has decided that this release schedule is too rapid. We need to provide more time for groups to adjust to the new specification before making incremental changes. Also, some of the original targeted development areas have been changed in the project timeline due to community interest and availability. For example, adding "Agronomy and Crop Modeling" concepts was moved up in the timeline as a few members of the community were working on a relevant project they wanted to push a new standard for, while the "Phenomics" concept might be delayed due to funding issues of key community members involved in making that specification. Goal 2 is currently behind schedule according to the original timeline. However, it seems clear that much of the work for this objective is front loaded. Once the first two tasks are complete, the remaining two tasks should be completed quickly. The task "Build an Interactive BrAPI Tutorial", from the original Objective 3, is starting a few months behind schedule as it was deemed lower priority than some of the other tasks in Objective 3 which have proceeded on time as planned. What opportunities for training and professional development has the project provided?Both the community hackathons and the new sabbatical program, discussed in Objective 4, have created opportunities for training and professional development for members of the BrAPI community. Each community hackathon begins with a formal "BrAPI 101" training class to bring everyone up to speed on what BrAPI is, how it works, and how to get started. This class can be requested for specific groups at any time throughout the year, but it will always be presented at least twice a year publicly for the hackathons. The hackathons and sabbaticals both provide opportunities to work with different groups who have different skills and specialties. This often provides informal training as different tools, ideas, and techniques are shared between groups. To formalize this further, one of the sabbatical teams this year had training built into their project goals. Along with the technical work, the traveler wanted to learn an application well enough that he could bring that knowledge back and train the rest of his organization. During the end of sabbatical demo, it was clear that he understood the tool very well and had made technical improvement to it. How have the results been disseminated to communities of interest?All news, materials, events, tools and updates in the BrAPI project are communicated to the BrAPI community through the community mailing list, the community Slack workspace, and posted on the project website (brapi.org). The community hackathons are also used as a forum to directly discuss new ideas, gather community feedback, and present important project results. Sometimes, large or important results are communicated to other relevant communities through conference and webinar presentations. What do you plan to do during the next reporting period to accomplish the goals? Complete separation of BrAPI Data Model from the RESTful documentation Develop a standard GraphQL BrAPI implementation Develop best practice documentation and standards for the BrAPI specification Create a set of self-guided, interactive, training material Write and submit a new manuscript outlining the updates in the BrAPI project since 2018 Organize two hackathons Conduct another round of the BrAPI Sabbatical program Continue attending and presenting at relevant conferences
Impacts
What was accomplished under these goals?
Goal 1: The BrAPI specification version V2.1 was officially released in July of 2022. BrAPI V2.1 addresses over 50 problems and enhancements to the specification requested by the BrAPI community. Community requests are tracked through the GitHub issues page and prioritized by general interest and comments left on each issue by members of the community. Issues can vary in size and complexity, ranging from simple spelling errors to the addition of new concepts into the specification. V2.1 introduced 22 new endpoints and 4 new concepts to the specification. These concepts include new data models for pedigree data, sample plate data, agronomic event data, and efficient allele matrix data. Two new working groups were established for looking into specific enhancements for the next version of the BrAPI specification. The first working group is focused on a GraphQL implementation of BrAPI. The second working group is focused on general analysis processes and how to connect with the existing BrAPI specification. This work will make it into the next version BrAPI as a new set of data models and best practice documentation. Goal 2: A new BrAPI community working group was established, focused on building a standard GraphQL implementation of BrAPI. This group has met regularly for the past six months to determine the changes needed to accommodate a GraphQL implementation. The first major task the group identified and began working on involves the extraction of the BrAPI data model schema from the RESTful API documentation. An independent data model schema will define the BrAPI data standard outside of the context of a RESTful API. The standard data models can then be used easily in many other contexts including a GraphQL API system, a database implementation, a message queue system, or any other context where breeding data needs to be represented in transit or at rest. Goal 3: Following the release of BrAPI V2.1, some of the key pieces of the BrAPI project infrastructure needed to be upgraded to match the new specification. The BrAPI Test Server is a public example implementation of the standard with test data allowing the community to test their BrAPI compatible client applications. The Test Server was upgraded to be compatible with V2.1, while maintaining compatibility with all previous versions of the specification. The BRAVA testing tool is an application which can be used to test any BrAPI compatible server. BRAVA was also upgraded to support V2.1, and BRAVA and the Test Server were used against each other to confirm full testing compliance. A number of bug fixes and small enhancements were made to both applications. When both of these tools were fully upgraded, the community could begin their own development efforts with full testing support. During Q1 2023, a number of updates were made to the community website brapi.org. Beyond general maintenance and content updates, three new tools were introduced to help the community. The BrAPI Servers page allows groups to register their BrAPI compatible servers and data sources to be advertised on the website. This tool makes it easy to find other data sources within the community to collaborate with, share data, and test functionality. Similarly, the BrAPI Compatible Software page was also introduced. This page allows community members to register and promote software that is BrAPI compatible, making it easier to share tools and applications with other members of the community. These two pages clearly show the success of the BrAPI project so far by advertising how the community is able to share data, computational resources, and tools using the BrAPI specification. The third tool introduced was the BrAPI Glossary. This is a complete list of all terms used in BrAPI, their definitions, and links to related terms in other standards and ontologies. More enhancements are expected for this tool to better connect it to the independent BrAPI data model described inObjective 2. In addition to these new tools, there were two new informational pages added to the website. The BrAPI Sabbaticals page documents the new sabbatical program (discussed more inObjective 4). This page explains the general information about the program, rules and expectations for applying, and templates to get started with the documentation. It also includes all the documentation and demo recordings from past sabbatical projects. The BrAPI Governance page documents the official governance of the BrAPI project. This includes information on the day-to-day project management team, the community advisory board, and some basic bylaws for nominating and electing advisory board members. This formal governance documentation represents a key piece of the management plan described in the proposal for this project. Goal 4: Two BrAPI community hackathons were executed this year. The first, on October 24-28 2022, was completely virtual. There were 45 participants representing 20 different organizations from around the world, and approximately one third of the attendees were new to the BrAPI community. Many of the projects were focused on upgrading existing implementations. There was a lot of collaboration between groups as they figured out the details of the new specification, with a particular focus on the new pedigree and allele matrix concepts. The second hackathon, was a mixed virtual and in-person event hosted by Clemson University. This was the first in-person BrAPI hackathon in over three years. This was a smaller event than past in-person hackathons, with 20 attendees. Approximately one quarter of the attendees were new to BrAPI. The lower attendance rate was expected as the world continues to recover from the stress of the pandemic. We hope to see more of the community in-person at the planned spring 2024 hackathon. The new BrAPI Sabbaticals Program was established. The sabbaticals are longer form collaboration efforts between groups, where a developer travels to a different organization for up to 6 weeks and engages in an in-person development project. As planned, this new program was able to fund the travel for two teams, one traveler from the US working with a team in France, and another going from Kenya to work with a team in Australia. The project teams were very successful, both with their technical achievements and their international collaboration and team building achievements. The teams have plans to continue working on their projects and maintain the collaborative effort. The final demo for both projects was presented virtually in June 2023, and the call for the next round of project proposals is now open. We feel confident in our ability to repeat the success of this new program next year. Goal 5: Along with the community hackathons, discussed inObjective 4, conferences and webinars are a great way to introduce new people to BrAPI and grow the community. In the past year, the BrAPI project coordinator has presented BrAPI at the following events: the EMBL-EBI 2023 Plant Webinar Series, the AGBT Ag Conference, the NAPDC conference, and the AgBioData Monthly Webinar. Each of these events targeted a different community within the domain of international agriculture and breeding. Each event led to follow-up conversations from at least one attendee/group. This year BrAPI was provided letters of support/collaboration for two new grant proposals regarding high dimensional omics data and genebank ordering systems, respectively. Additionally, BrAPI joined the group of an existing grant (Dsfas-Cin: Integrating Multiscale Remote Sensing Data for Enhancing Data-Driven Predictive Analytics in Crop Breeding and Management) to standardize drone imaging support. If successful, all three of these efforts will increase the size and diversity of the BrAPI community while adding new functionality to the specification.
Publications
|