nf-core/configs: ebi_codon_slurm
The European Bioinformatics Institute HPC cluster (codon) profile for the SLURM login nodes
EBI Codon Cluster SLURM Configuration
All nf-core pipelines have been successfully configured for use on the SLURM login nodes of the codon cluster at the European Bioinformatics Institute.
To use, run the pipeline with -profile ebi_codon_slurm. This will download and launch the ebi_codon_slurm.config which has been pre-configured with a setup suitable for the codon cluster.
You should not run Nextflow on the login nodes. You should submit a batch job that executes Nextflow.
Loading the required modules
Before running the pipeline you will need to load Nextflow and Singularity using the environment module system on the codon cluster. You can do this by issuing the commands below:
## Load Nextflow and Singularity environment modules
module purge
module load nextflow/22.10.1
module load singularityce/3.10.3You may want to add those module load commands to your shell configuration file if you use them often.
Installing mamba
Run the following:
curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh"
bash Mambaforge-$(uname)-$(uname -m).shFollow the instructions here for more details.
Setting up a suitable path for the Nextflow software cache
It is recommended to install conda environments and singularity containers in your /hps/software directory. To achieve this, add to your ~/.nextflow/config file the following lines:
singularity.cacheDir = "/hps/software/users/<group>/<user_id>/nextflow_software_cache/singularity"
conda.cacheDir = "/hps/software/users/<group>/<user_id>/nextflow_software_cache/conda"Config file
/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Nextflow config file for EMBL-EBI Codon Cluster for the SLURM login nodes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Author: Saul Pierotti
Mail: saul@ebi.ac.uk
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*/
params {
config_profile_contact = "Saul Pierotti (@saulpierotti)"
config_profile_description = "The European Bioinformatics Institute HPC cluster (codon) profile for the SLURM login nodes"
config_profile_url = "https://www.ebi.ac.uk/"
}
singularity {
enabled = true
// the default is 20 minutes and fails with large images
pullTimeout = "3 hours"
}
process{
// this is to avoid errors for missing files due to shared filesystem latency
maxRetries = 3
errorStrategy = { ( task.exitStatus == 0 ) ? "retry" : "terminate" }
cache = "lenient"
afterScript = "sleep 60" // to avoid fail when using storeDir for missing output
}
executor {
name = "slurm"
queueSize = 2000
submitRateLimit = "10/1sec"
exitReadTimeout = "30 min"
jobName = {
task.name // [] and " " not allowed in lsf job names
.replace("[", "(")
.replace("]", ")")
.replace(" ", "_")
}
}
homepage
contact
