A New Chapter for Benchmarking SIG
Benchmarking is an essential part of bioinformatics, but setting up fair, reproducible, and standardized comparisons is often a massive challenge. To tackle this together, we are officially activating the Benchmarking Special Interest Group. We are kicking things off with a brand new series of monthly meetings and regular blog updates to keep the community connected.
Our vision for this group is built around three main ideas. First, we want to help developers by building modular benchmarking components in nf-core so that validation steps are simple to plug and play into any pipeline. Second, we want to act as a community forum to find out exactly what researchers need to compare, whether that is metagenomics, spatial transcriptomics, or drug response prediction. Finally, we want to make sure our efforts stay aligned with external standards and platforms like NCBench and openEBench.
Join Our First Monthly Meeting
To get the momentum going, our very first monthly meeting will focus on the nf-core/variantbenchmarking pipeline. Main developer of the pipeline, @kubranarci, will be sharing some insights on classical variant benchmarking and might run through a quick live demo.
However, the presentation is only half the event. The real purpose of this first meeting is to bring everyone together for an open roundtable discussion. We want to hear directly from you. This is the perfect time to talk about pipeline feature requests, share new ideas, and explore the broader problems and solutions you face in variant benchmarking.
What to Expect Next
While we are starting with variant benchmarking, our scope is completely open to any type of method comparison, validation, or resource efficiency study.
Going forward, our monthly meetings will follow a split format: a 30 minute presentation followed by a 30 minute open discussion. We are lining up a diverse range of topics for the future, including deep dives into benchmarking projects, reviews of new software tools, infrastructure benchmarking, and inspiring talks from leading genomics researchers. You can explore our full roadmap on the main SIG page.
Get Involved
We invite all developers, researchers, and benchmarking enthusiasts to join the conversation. Please join the nf-core Slack workspace and say hello in the #benchmarking channel. We would love to hear your ideas, suggest meeting topics, and work together to shape the future of benchmarking in our community.
