CSIRO Petrichor HPC Configuration

To run an nf-core pipeline at CSIRO Petrichor HPC, execute with -profile singularity,csiro_petrichor. This will download and launch the csiro_petrichor.config which has been pre-configured with a setup suitable for the CSIRO Petrichor HPC cluster. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a Singularity image before execution of the pipeline.

Access to CSIRO Petrichor HPC

Please be aware that you will need to have an active CSIRO HPC User Account in order to use this infrastructure. If you need help getting started, please see the guide for new users.

Launch an nf-core pipeline on Petrichor

Prerequisites

Before running the pipeline you will need to load Nextflow and Singularity, both of which are globally installed modules. You can do this by running the commands below:

module purge
module load nextflow singularity

Cluster considerations

Before running an nf-core pipeline you will need to set your project code as a Slurm environment variable:

export SBATCH_ACCOUNT=OD-012345

You can retrieve a list of projects your ident is associate with using:

get_project_codes

Execution command

module load nextflow
module load singularity
 
nextflow run <nf-core_pipeline>/main.nf \
  -profile singularity,csiro_petrichor \
  <additional flags>

Config file

See config file on GitHub

csiro_petrichor.config
// CSIRO Petrichor nf-core configuration profile
params {
  config_profile_description = 'CSIRO Petrichor HPC profile provided by nf-core/configs'
  config_profile_contact = 'Mitchell OBrien (@mitchob)'
  config_profile_url = 'https://confluence.csiro.au/display/SC/CSIRO+SC+Shared+Cluster+-+Petrichor'
}
 
// Enable use of Singularity to run containers
singularity {
  enabled = true
  autoMounts = true
  autoCleanUp = true
}
 
// Submit up to XX concurrent jobs 
//executor {
//  queueSize = XX
//}
 
// Define process resource limits
process {
  executor = 'slurm'
  clusterOptions = "--account=${System.getenv('SBATCH_ACCOUNT')}"
  module = 'singularity/3.8.7'
  cache = 'lenient'
  stageInMode = 'symlink'
  queue = 'defq'
}