Prince Configuration
nf-core pipelines that use this repo
All nf-core pipelines that use this config repo (which is most), can be run on prince. Before running a pipeline for the first time, go into an interactive slurm session on a compute node (srun --pty --time=02:00:00 -c 2), as the docker image for the pipeline will need to be pulled and converted.
Now, run the pipeline of your choice with -profile prince. This will download and launch the prince.config which has been pre-configured with a setup suitable for the prince cluster. Using this profile, a docker image containing all of the required software will be downloaded, and converted to a singularity image before execution of the pipeline. This step takes time!!
An example commandline:
nextflow run nf-core/<pipeline name> -profile prince <additional flags>
nf-core pipelines that do not use this repo
If the pipeline has not yet been configured to use this config, then you will have to do it manually.
git clone this repo, copy the prince.config from the conf folder and then you can invoke the pipeline like this:
nextflow run nf-core/<pipeline name> -c prince.config <additional flags>
NB: You will need an account to use the HPC cluster Prince in order to run the pipeline. If in doubt contact the HPC admins. NB: Rather than using the nextflow module, I recommend you install nextflow in your home directory - instructions are on nextflow.io (or ask the writer of this profile). The reason this is better than using the module for nextflow on the cluster, is that the development cycle of nextflow is rapid and it’s easy to update your installation yourself:
nextflow self-update.
Config file
def singularityDir = set_singularity_path()
def singularityModule = "singularity/3.2.1"
def squashfsModule = "squashfs/4.3"
params {
config_profile_description = """
NYU prince cluster profile provided by nf-core/configs.
Run from your scratch directory, the output files may be large!
Please consider running the pipeline on a compute node the first time, as it will be pulling the docker image, which will be converted into a singularity image, which is heavy on the login node. Subsequent runs can be done on the login node, as the docker image will only be pulled and converted once. By default the images will be stored in $singularityDir
""".stripIndent()
config_profile_contact = 'Tobias Schraink (@tobsecret)'
config_profile_url = 'https://github.com/nf-core/configs/blob/master/docs/prince.md'
}
singularity {
enabled = true
cacheDir = singularityDir
}
process {
beforeScript = "module load $singularityModule $squashfsModule"
executor = 'slurm'
}
def set_singularity_path() {
def scratch = System.getenv('SCRATCH')
if(scratch == null){
System.err.println("WARNING: prince.config requires the SCRATCH env var to be set.")
return null
} else {
return "$SCRATCH/singularity_images_nextflow"
}
}
def singularityDir = set_singularity_path()
def singularityModule = "singularity/3.2.1"
def squashfsModule = "squashfs/4.3"
params {
config_profile_description = """
NYU prince cluster profile provided by nf-core/configs.
Run from your scratch directory, the output files may be large!
Please consider running the pipeline on a compute node the first time, as it will be pulling the docker image, which will be converted into a singularity image, which is heavy on the login node. Subsequent runs can be done on the login node, as the docker image will only be pulled and converted once. By default the images will be stored in $singularityDir
""".stripIndent()
config_profile_contact = 'Tobias Schraink (@tobsecret)'
config_profile_url = 'https://github.com/nf-core/configs/blob/master/docs/prince.md'
}
singularity {
enabled = true
cacheDir = singularityDir
}
process {
beforeScript = "module load $singularityModule $squashfsModule"
executor = 'slurm'
}
def set_singularity_path() {
def scratch = System.getenv('SCRATCH')
if(scratch == null){
System.err.println("WARNING: prince.config requires the SCRATCH env var to be set.")
return null
} else {
return "$SCRATCH/singularity_images_nextflow"
}
}
executor
contact
