nf-core/configs: vai
Van Andel Institute HPC profile provided by nf-core/configs.
VAI configuration
All nf-core pipelines have been successfully configured for use on the HPC cluster at Van Andel Institute.
To use, run the pipeline with -profile vai. This will download and launch the vai.config which has been pre-configured with a setup suitable for the VAI HPC. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Singularity image before execution of the pipeline.
module load singularity
NXF_OPTS="-Xmx500m" MALLOC_ARENA_MAX=4 nextflow run <pipeline>
NB: You will need an account to use the HPC in order to run the pipeline. If in doubt contact IT.
NB: Nextflow will need to submit the jobs via the job scheduler to the HPC cluster and as such the commands above will have to be executed on the login node. If in doubt contact IT.
NB: The submit node limits the amount of memory available to each user. TheNXF_OPTSandMALLOC_ARENA_MAXparameters above prevent Nextflow from allocating more memory than the scheduler will allow.
Config file
params {
config_profile_description = 'Van Andel Institute HPC profile provided by nf-core/configs.'
config_profile_contact = 'Nathan Spix (@njspix)'
config_profile_url = 'https://vanandelinstitute.sharepoint.com/sites/SC/SitePages/HPC3-High-Performance-Cluster-and-Cloud-Computing.aspx'
max_memory = 250.GB
max_cpus = 40
max_time = 336.h
}
singularity {
enabled = true
autoMounts = true
}
// See if we can find evidence that we're on the new HPC
def new_cluster = false
try {
new_cluster = ['/bin/bash', '-c', 'echo $HOSTNAME'].execute().text.trim() ==~ /(compute|submit)[0-9]{3}/
} catch (java.io.IOException e) {
System.err.println("WARNING: Couldn't figure out which cluster we're on, defaulting to old (PBS) cluster")
}
if (new_cluster) {
System.out.println("Using VAI institutional configuration for new HPC cluster")
} else {
System.out.println("Using VAI institutional configuration for old HPC cluster")
}
process {
beforeScript = { new_cluster ? '' : 'module load VARI/singularity' }()
executor = { new_cluster ? 'slurm' : 'pbs' }()
queue = { new_cluster ? 'long' : { task.time <= 48.h ? 'shortq' : 'longq' } }()
}
executor
contact
