Resource usage guidance

nf-core/mag is a bioinformatics best-practice analysis pipeline for the assembly, binning, and annotation of metagenomes. Due to the nature of metagenomic samples, nf-core/mag can in some cases require large amounts of computational resources to execute. In this page we provide information and guidance on how nf-core/mag works by default, and in some cases how to optimise computational resource usage.

Multi-sample or bin executing modules

By default, nf-core/mag aims to make the runtime as efficiently as possible on HPC and similar infrastructure through generating more but shorter running jobs.

In most cases, all pre-assembly steps typically run one job per input FASTQ file. After assembly, most steps run on a per assembly or contig. After binning, each step runs on a per-bin basis, and so forth.

There are a few exceptions to this. In the following cases, depending on the options you set in the pipeline, you may see some steps running as a single job with multiple samples.

• Multiple samples processed in one job:
  • If --coassemble_group: all reads across all samples in a group will be pooled into one assembly job (e.g. MEGAHIT, metaSPAdes)
  • If --gtdbtk_single_job: all bins across all samples will be pooled into a single GTDBTk classification job

Default resource requests

The following table lists the default resources that Nextflow will request from a machine on it’s first execution attempt for a given module. It summaries information recorded in the file conf/base.conf and the module files themselves modules.

To customise these resource requests, see the central nf-core instructions.

Please also note the following:

• Not all modules in this table will run in every pipeline run, nor will the tool necessarily use the requested amount.
• On certain module execution failures (such as out of memory), Nextflow will try and resubmit with increased resources with the equation <resource value> * task.attempt, up to a certain number of retries. See conf/base.conf for more information.
• Some modules have a dedicate flag to fix the number of CPUs for reproducibility reasons (e.g. --megahit_fix_cpu_1). See the parameters page.
• METASPADES, METASPADESHYBRID, FILTLONG, METAMDBG_ASM and FLYE grow memory and time exponentially with each retry (<value> * 2 ** (task.attempt - 1)); the table shows the first-attempt request.
• ALE requests 18.GB by default, but 30.GB when --coassemble_group is set.

The table is ordered first by ‘Source’ of defaults by specificity of definition, then by ‘memory’, and then ‘Module name’ by alphabetical order.

Table generated for nf-core/mag v5.5, using Claude Haiku 4.5, and corrected for accuracy by a human.