MPCDF Configuration

All nf-core pipelines have been successfully configured for use on the HPCs at Max Planck Computing and Data Facility.

⚠️ these profiles are not officially supported by the MPCDF.

To run Nextflow, the jdk module must be loaded. To use the nf-core profile(s), run the pipeline with -profile mpcdf,<cluster>.

Currently profiles for the following clusters are supported: raven.

For viper, please see the mpcdf_viper profile in the MPCDF Viper Configuration.

All profiles use singularity as the corresponding containerEngine. To prevent repeatedly downloading the same singularity image for every pipeline run, for all profiles we recommend specifying a cache location in your ~/.bash_profile with the $NXF_SINGULARITY_CACHEDIR bash variable.

NB: Nextflow will need to submit the jobs via SLURM to the clusters and as such the commands above will have to be executed on one of the head nodes. Check the MPCDF documentation.

Global Profiles

raven

To use: -profile mpcdf,raven

Sets the following parameters:

  • Maximum parallel running jobs: 8
  • Max. memory: 2000000.MB (2.TB)
  • Max. CPUs: 72
  • Max. walltime: 24.h

Config file

See config file on GitHub

mpcdf.config
params {
    config_profile_description = 'MPCDF HPC profiles (unoffically) provided by nf-core/configs.'
    config_profile_contact     = 'James Fellows Yates (@jfy133)'
    config_profile_url         = 'https://www.mpcdf.mpg.de/services/supercomputing'
}
 
profiles {
 
    raven {
 
        cleanup = true
 
        process {
            resourceLimits = [
                memory: 2000000.MB,
                cpus: 72,
                time: 24.h
            ]
            beforeScript = 'module load singularity'
            executor     = 'slurm'
        }
 
        executor {
            queueSize         = 30
            pollInterval      = '1 min'
            queueStatInterval = '5 min'
        }
 
        // Set $NXF_SINGULARITY_CACHEDIR in your ~/.bash_profile
        // to stop downloading the same image for every run
        singularity {
            enabled    = true
            autoMounts = true
        }
 
        params {
            config_profile_description = 'MPCDF raven profile (unofficially) provided by nf-core/configs.'
            max_memory                 = 2000000.MB
            max_cpus                   = 72
            max_time                   = 24.h
        }
    }
 
    debug {
        cleanup = false
    }
}