MPCDF Configuration
All nf-core pipelines have been successfully configured for use on the HPCs at Max Planck Computing and Data Facility.
⚠️ these profiles are not officially supported by the MPCDF.
To run Nextflow, the jdk module must be loaded. To use the nf-core profile(s), run the pipeline with -profile mpcdf,<cluster>.
Currently profiles for the following clusters are supported: cobra, raven
All profiles use singularity as the corresponding containerEngine. To prevent repeatedly downloading the same singularity image for every pipeline run, for all profiles we recommend specifying a cache location in your ~/.bash_profile with the $NXF_SINGULARITY_CACHEDIR bash variable.
NB: Nextflow will need to submit the jobs via SLURM to the clusters and as such the commands above will have to be executed on one of the head nodes. Check the MPCDF documentation.
Global Profiles
cobra
To use: -profile mpcdf,cobra
Sets the following parameters:
- Maximum parallel running jobs: 8
- Max. memory: 725.GB
- Max. CPUs: 40
- Max. walltime: 24.h
raven
To use: -profile mpcdf,raven
Sets the following parameters:
- Maximum parallel running jobs: 8
- Max. memory: 2000000.MB (2.TB)
- Max. CPUs: 72
- Max. walltime: 24.h
Pipeline Profiles
nf-core/eager
We offer a pipeline specific profile optimised for nf-core/eager
- Maximum parallel running jobs: 30
nf-core/rnaseq
We offer a pipeline specific profile optimised for nf-core/rnaseq
Config file
params {
config_profile_description = 'MPCDF HPC profiles (unoffically) provided by nf-core/configs.'
config_profile_contact = 'James Fellows Yates (@jfy133)'
config_profile_url = 'https://www.mpcdf.mpg.de/services/supercomputing'
}
profiles {
cobra {
cleanup = true
process {
beforeScript = 'module load singularity'
executor = 'slurm'
}
executor {
queueSize = 8
pollInterval = '1 min'
queueStatInterval = '5 min'
}
// Set $NXF_SINGULARITY_CACHEDIR in your ~/.bash_profile
// to stop downloading the same image for every run
singularity {
enabled = true
autoMounts = true
}
params {
config_profile_description = 'MPCDF cobra profile (unofficially) provided by nf-core/configs.'
max_memory = 725.GB
max_cpus = 40
max_time = 24.h
}
}
raven {
cleanup = true
process {
beforeScript = 'module load singularity'
executor = 'slurm'
}
executor {
queueSize = 30
pollInterval = '1 min'
queueStatInterval = '5 min'
}
// Set $NXF_SINGULARITY_CACHEDIR in your ~/.bash_profile
// to stop downloading the same image for every run
singularity {
enabled = true
autoMounts = true
}
params {
config_profile_description = 'MPCDF raven profile (unofficially) provided by nf-core/configs.'
max_memory = 2000000.MB
max_cpus = 72
max_time = 24.h
}
}
debug {
cleanup = false
}
}
executor
contact
