nf_core.download

Downloads a nf-core pipeline to the local file system.

classnf_core.download.DownloadProgress(*columns: str | ProgressColumn, console: Console | None = None, auto_refresh: bool = True, refresh_per_second: float = 10, speed_estimate_period: float = 30.0, transient: bool = False, redirect_stdout: bool = True, redirect_stderr: bool = True, get_time: Callable[[], float] | None = None, disable: bool = False, expand: bool = False)

Bases: Progress

Custom Progress bar class, allowing us to have two progress bars with different columns / layouts.

get_renderables()

Get a number of renderables for the progress display.

classnf_core.download.DownloadWorkflow(pipeline=None, revision=None, outdir=None, compress_type=None, force=False, container=None, singularity_cache_only=False, parallel_downloads=4)

Bases: object

Downloads a nf-core workflow from GitHub to the local file system.

Can also download its Singularity container image if required.

  • Parameters:
    • pipeline (str) – A nf-core pipeline name.
    • revision (str) – The workflow revision to download, like 1.0. Defaults to None.
    • singularity (bool) – Flag, if the Singularity container should be downloaded as well. Defaults to False.
    • outdir (str) – Path to the local download directory. Defaults to None.

compress_download()

Take the downloaded files and make a compressed .tar.gz archive.

download_configs()

Downloads the centralised config profiles from nf-core/configs to self.outdir.

download_wf_files()

Downloads workflow files from GitHub to the self.outdir.

download_workflow()

Starts a nf-core workflow download.

find_container_images()

Find container image names for workflow.

Starts by using nextflow config to pull out any process.container declarations. This works for DSL1. It should return a simple string with resolved logic.

Second, we look for DSL2 containers. These can’t be found with nextflow config at the time of writing, so we scrape the pipeline files. This returns raw source code that will likely need to be cleaned.

If multiple containers are found, prioritise any prefixed with http for direct download.

Example syntax:

Early DSL2: : if (workflow.containerEngine == ‘singularity’ && !params.singularity_pull_docker_container) { : container “https://depot.galaxyproject.org/singularity/fastqc:0.11.9–0”
} else { : container “quay.io/biocontainers/fastqc:0.11.9–0”
}

Later DSL2: : container “${ workflow.containerEngine == ‘singularity’ && !task.ext.singularity_pull_docker_container ? : ‘https://depot.galaxyproject.org/singularity/fastqc:0.11.9–0’ : ‘quay.io/biocontainers/fastqc:0.11.9–0’ }”

DSL1 / Special case DSL2: : container “nfcore/cellranger:6.0.2”

get_revision_hash()

Find specified revision / branch hash

get_singularity_images()

Loop through container names and download Singularity images

prompt_compression_type()

Ask user if we should compress the downloaded files

prompt_container_download()

Prompt whether to download container images or not

prompt_pipeline_name()

Prompt for the pipeline name if not set with a flag

prompt_revision()

Prompt for pipeline revision / branch

prompt_singularity_cachedir_only()

Ask if we should only use $NXF_SINGULARITY_CACHEDIR without copying into target

prompt_use_singularity_cachedir()

Prompt about using $NXF_SINGULARITY_CACHEDIR if not already set

singularity_copy_cache_image(container, out_path, cache_path)

Copy Singularity image from NXF_SINGULARITY_CACHEDIR to target folder.

singularity_download_image(container, out_path, cache_path, progress)

Download a singularity image from the web.

Use native Python to download the file.

  • Parameters:
    • container (str) – A pipeline’s container name. Usually it is of similar format to https://depot.galaxyproject.org/singularity/name:version
    • out_path (str) – The final target output path
    • cache_path (str*,* None) – The NXF_SINGULARITY_CACHEDIR path if set, None if not
    • progress (Progress) – Rich progress bar instance to add tasks to.

singularity_image_filenames(container)

Check Singularity cache for image, copy to destination folder if found.

  • Parameters: container (str) – A pipeline’s container name. Can be direct download URL or a Docker Hub repository ID.
  • Returns: Returns True if we have the image in the target location. : Returns a download path if not.
  • Return type: results (bool, str)

singularity_pull_image(container, out_path, cache_path, progress)

Pull a singularity image using singularity pull

Attempt to use a local installation of singularity to pull the image.

  • Parameters: container (str) – A pipeline’s container name. Usually it is of similar format to nfcore/name:version.
  • Raises: Various exceptions possible from subprocess execution of Singularity. –

wf_use_local_configs()

Edit the downloaded nextflow.config file to use the local config files
On this page