nf-core/configs: utd_ganymede
University of Texas at Dallas HPC cluster profile provided by nf-core/configs
UTD Ganymede Configuration
All nf-core pipelines have been successfully configured for use on the Ganymede HPC cluster at The Univeristy of Texas at Dallas.
To use, run the pipeline with -profile utd_ganymede. This will download and launch the utd_ganymede.config which has been pre-configured with a setup suitable for Ganymede.
Before running the pipeline you will need to load Singularity using the environment module system on Ganymede. You can do this by issuing the commands below:
## Singularity environment modules
module purge
module load singularityAll of the intermediate files required to run the pipeline will be stored in the work/ directory. It is recommended to delete this directory after the pipeline has finished successfully because it can get quite large, and all of the main output files will be saved in the results/ directory anyway.
[!NOTE] You will need an account to use the HPC cluster on Ganymede in order to run the pipeline. https://docs.circ.utdallas.edu/user-guide/accounts/index.html Nextflow will need to submit the jobs via SLURM to the HPC cluster and as such the commands above will have to be executed on the login node. If in doubt contact CIRC.
Config file
//Profile config names for nf-core/configs
params {
config_profile_description = 'University of Texas at Dallas HPC cluster profile provided by nf-core/configs'
config_profile_contact = 'Edmund Miller'
config_profile_contact_github = '@edmundmiller'
config_profile_contact_email = 'edmund.miller@utdallas.edu'
config_profile_url = 'https://docs.circ.utdallas.edu/user-guide/systems/ganymede.html'
}
env {
TMPDIR = "/home/$USER/scratch/tmp"
SINGULARITY_CACHEDIR="/home/$USER/scratch/singularity"
}
singularity {
enabled = true
envWhitelist='SINGULARITY_BINDPATH,LD_LIBRARY_PATH'
autoMounts = true
cacheDir = "/home/$USER/scratch/singularity"
}
def membership = "groups".execute().text
def select_queue = { memory, cpu ->
if (memory <= 30.GB && cpu <= 16 && membership.contains('genomics')) {
return 'genomics,normal'
}
if (memory > 30.GB && memory <= 125.GB && cpu <= 12 && membership.contains('kim')) {
return 'Kim,128s'
}
if (memory > 30.GB && memory <= 125.GB && cpu <= 16) {
return '128s'
}
if (memory <= 250.GB && cpu <= 28) {
return '256i,256h'
}
}
// Submit up to 100 concurrent jobs
// pollInterval and queueStatInterval of every 5 minutes
// submitRateLimit of 20 per minute
executor {
queueSize = 100
pollInterval = '5 min'
queueStatInterval = '5 min'
submitRateLimit = '20 min'
jobName = { "${task.process.split(':').last()}" }
}
process {
beforeScript = 'module load singularity/3.2.1'
executor = 'slurm'
queue = { select_queue(task.memory, task.cpu) }
withLabel:process_medium {
cpus = { check_max( 16 * task.attempt, 'cpus' ) }
memory = { check_max( 30.GB * task.attempt, 'memory' ) }
}
}
params {
max_memory = 250.GB
max_cpus = 28
max_time = 96.h
}
executor
contact
