University of Washington Hyak Profile for the Department of Pediatrics

All nf-core pipelines have been successfully configured for use on the University of Washington’s Hyak cluster, with this profile specific to the Department of Pediatrics partition.

To use, run the pipeline with -profile uw_hyak_pedslabs. This will download and launch the uw_hyak_pedslabs.config which has been pre-configured with a setup suitable for the Hyak cluster. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Singularity image before execution of the pipeline.

Running the workflow on the Pasteur cluster

Nextflow is not installed by default on Hyak.

  • Install Nextflow : here

Nextflow manages each process as a separate job that is submitted to the cluster by using the sbatch command. Nextflow shouldn’t run directly on the submission node but on a compute node.

To install and use Nextflow, the following process can be used:

  1. Install mamba and create a mamba environment containing nextflow and nf-core
wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh
bash Mambaforge-Linux-x86_64.sh -b -p $HOME/mambaforge
mamba create -n nextflow -c bioconda -c conda-forge python=3.8 nf-core nextflow -y -q
wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh
bash Mambaforge-Linux-x86_64.sh -b -p $HOME/mambaforge
mamba create -n nextflow -c bioconda -c conda-forge python=3.8 nf-core nextflow -y -q
  1. Run nextflow on a compute node:
# create a screen
screen -S nextflow
# request a compute node (mem and time requests can be modified)
salloc -A pedslabs -p compute-hugemem -N 1 -c 1 --mem=16GB --time=12:00:00
# load the nextflow environment
mamba activate nextflow
# Run nextflow workflow
nextflow run \\
<nf-core-pipeline-name> \\
-resume
-profile uw_hyak_pedslabs \\
--email <uw-net-id>@uw.edu
# create a screen
screen -S nextflow
# request a compute node (mem and time requests can be modified)
salloc -A pedslabs -p compute-hugemem -N 1 -c 1 --mem=16GB --time=12:00:00
# load the nextflow environment
mamba activate nextflow
# Run nextflow workflow
nextflow run \\
<nf-core-pipeline-name> \\
-resume
-profile uw_hyak_pedslabs \\
--email <uw-net-id>@uw.edu

Config file

See config file on GitHub

uw_hyak_pedslabs.config
params {
  config_profile_description = 'UW Hyak Pedslabs cluster profile provided by nf-core/configs.'
  config_profile_contact = 'Carson J. Miller (@CarsonJM)'
  config_profile_url = 'https://www.peds.uw.edu/'
  max_memory = 742.GB
  max_cpus = 40
  max_time = 72.h
}
 
process {
    executor = 'slurm'
    queue = { task.attempt == 1 ? 'ckpt' : 'compute-hugemem' }
    maxRetries = 1
    clusterOptions = { "-A pedslabs" }
    scratch = '/gscratch/scrubbed/pedslabs/'
}
 
executor {
    queuesize = 50
    submitRateLimit = '1 sec'
}
 
singularity {
  enabled = true
  autoMounts = true
  cacheDir = '/gscratch/scrubbed/pedslabs/.apptainer'
}
 
debug {
  cleanup = false
}
 
uw_hyak_pedslabs.config
params {
  config_profile_description = 'UW Hyak Pedslabs cluster profile provided by nf-core/configs.'
  config_profile_contact = 'Carson J. Miller (@CarsonJM)'
  config_profile_url = 'https://www.peds.uw.edu/'
  max_memory = 742.GB
  max_cpus = 40
  max_time = 72.h
}
 
process {
    executor = 'slurm'
    queue = { task.attempt == 1 ? 'ckpt' : 'compute-hugemem' }
    maxRetries = 1
    clusterOptions = { "-A pedslabs" }
    scratch = '/gscratch/scrubbed/pedslabs/'
}
 
executor {
    queuesize = 50
    submitRateLimit = '1 sec'
}
 
singularity {
  enabled = true
  autoMounts = true
  cacheDir = '/gscratch/scrubbed/pedslabs/.apptainer'
}
 
debug {
  cleanup = false
}