Linting Errors

This page contains detailed descriptions of the tests done by the nf-core/tools package. Linting errors should show URLs next to any failures that link to the relevant heading below.

Error #1 - File not found / must be removed

nf-core pipelines should adhere to a common file structure for consistency.

The lint test looks for the following required files:

The following files are suggested but not a hard requirement. If they are missing they trigger a warning:

The following files will cause a failure if the are present (to fix, delete them):

Error #2 - Docker file check failed

Pipelines should have a files called Dockerfile in their root directory. The file is used for automated docker image builds. This test checks that the file exists and contains at least the string FROM (Dockerfile).

Error #3 - Licence check failed

nf-core pipelines must ship with an open source MIT licence.

This test fails if the following conditions are not met:

Error #4 - Nextflow config check failed

nf-core pipelines are required to be configured with a minimal set of variable names. This test fails or throws warnings if required variables are not set.

Note: These config variables must be set in nextflow.config or another config file imported from there. Any variables set in nextflow script files (eg. main.nf) are not checked and will be assumed to be missing.

The following variables fail the test if missing:

The following variables throw warnings if missing:

The following variables are depreciated and fail the test if they are still present:

Process-level configuration syntax is checked and fails if uses the old Nextflow syntax, for example: process.$fastqc instead of process withName:'fastqc'.

Error #5 - Continuous Integration configuration

nf-core pipelines must have CI testing with GitHub Actions.

GitHub Actions CI

There are 4 main GitHub Actions CI test files: ci.yml, linting.yml, branch.yml and awstests.yml, and they can all be found in the .github/workflows/ directory. You can always add steps to the workflows to suit your needs, but to ensure that the nf-core lint tests pass, keep the steps indicated here.

This test will fail if the following requirements are not met in these files:

  1. ci.yml: Contains all the commands required to test the pipeline

    • Must be triggered on the following events:

      on:
        push:
          branches:
            - dev
        pull_request:
        release:
          types: [published]
    • The minimum Nextflow version specified in the pipeline's nextflow.config has to match that defined by nxf_ver in the test matrix:

      strategy:
        matrix:
          # Nextflow versions: check pipeline minimum and current latest
          nxf_ver: ['19.10.0', '']
    • The Docker container for the pipeline must be tagged appropriately for:

      • Development pipelines: docker tag nfcore/<pipeline_name>:dev nfcore/<pipeline_name>:dev

      • Released pipelines: docker tag nfcore/<pipeline_name>:dev nfcore/<pipeline_name>:<tag>

        - name: Build new docker image
          if: env.GIT_DIFF
          run: docker build --no-cache . -t nfcore/<pipeline_name>:1.0.0
        
        - name: Pull docker image
          if: ${{ !env.GIT_DIFF }}
          run: |
            docker pull nfcore/<pipeline_name>:dev
            docker tag nfcore/<pipeline_name>:dev nfcore/<pipeline_name>:1.0.0
  2. linting.yml: Specifies the commands to lint the pipeline repository using nf-core lint and markdownlint

    • Must be turned on for push and pull_request.
    • Must have the command nf-core -l lint_log.txt lint ${GITHUB_WORKSPACE}.
    • Must have the command markdownlint ${GITHUB_WORKSPACE} -c ${GITHUB_WORKSPACE}/.github/markdownlint.yml.
  3. branch.yml: Ensures that pull requests to the protected master branch are coming from the correct branch when a PR is opened against the nf-core repository.

    • Must be turned on for pull_request to master.

      on:
        pull_request:
          branches:
          - master
    • Checks that PRs to the protected nf-core repo master branch can only come from an nf-core dev branch or a fork patch branch:

      steps:
        # PRs to the nf-core repo master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
        - name: Check PRs
          if: github.repository == 'nf-core/<pipeline_name>'
          run: |
            { [[ ${{github.event.pull_request.head.repo.full_name}} == nf-core/<pipeline_name> ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
    • For branch protection in repositories outside of nf-core, you can add an additional step to this workflow. Keep the nf-core branch protection step, to ensure that the nf-core lint tests pass. Here's an example:

      steps:
        # PRs are only ok if coming from an nf-core `dev` branch or a fork `patch` branch
        - name: Check PRs
          if: github.repository == 'nf-core/<pipeline_name>'
          run: |
            { [[ ${{github.event.pull_request.head.repo.full_name}} == nf-core/<pipeline_name> ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
        - name: Check PRs in another repository
          if: github.repository == '<repo_name>/<pipeline_name>'
          run: |
            { [[ ${{github.event.pull_request.head.repo.full_name}} == <repo_name>/<pipeline_name> ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
  4. awstest.yml: Triggers tests on AWS batch. As running tests on AWS incurs costs, they should be only triggered on push to master and release.

    • Must be turned on for push to master and release.
    • Must not be turned on for pull_request or other events.

GitHub Actions AWS full tests

Additionally, we provide the possibility of testing the pipeline on full size datasets on AWS. This should ensure that the pipeline runs as expected on AWS and provide a resource estimation. The GitHub Actions workflow is: awsfulltest.yml, and it can be found in the .github/workflows/ directory. This workflow incurrs higher AWS costs, therefore it should only be triggered on release. For tests on full data prior to release, [https://tower.nf](Nextflow Tower's launch feature) can be employed.

awsfulltest.yml: Triggers full sized tests run on AWS batch after releasing.

Error #6 - Repository README.md tests

The README.md files for a project are very important and must meet some requirements:

Error #7 - Pipeline and container version numbers

This test only runs when --release is set or $GITHUB_REF is equal to master

These tests look at process.container and $GITHUB_REF only if they are set.

Error #8 - Conda environment tests

These tests only run when your pipeline has a root file called environment.yml

Each dependency is checked using the Anaconda API service. Dependency sublists are ignored with the exception of - pip: these packages are also checked for pinned version numbers and checked using the PyPI JSON API.

Note that conda dependencies with pinned channels (eg. conda-forge::openjdk) are fine and should be handled by the linting properly.

Each dependency can have the following lint failures and warnings:

NB: Conda package versions should be pinned with one equals sign (toolname=1.1), pip with two (toolname==1.2)

Error #9 - Dockerfile for use with Conda environments

This test only runs if there is both environment.yml and Dockerfile present in the workflow.

If a workflow has a conda environment.yml file (see above), the Dockerfile should use this to create the container. Such Dockerfiles can usually be very short, eg:

FROM nfcore/base:1.7
MAINTAINER Rocky Balboa <your@email.com>
LABEL authors="your@email.com" \
    description="Docker image containing all requirements for the nf-core mypipeline pipeline"

COPY environment.yml /
RUN conda env create --quiet -f /environment.yml && conda clean -a
RUN conda env export --name nf-core-mypipeline-1.0 > nf-core-mypipeline-1.0.yml
ENV PATH /opt/conda/envs/nf-core-mypipeline-1.0/bin:$PATH

To enforce this minimal Dockerfile and check for common copy+paste errors, we require that the above template is used. Failures are generated if the FROM, COPY and RUN statements above are not present. These lines must be an exact copy of the above example.

Note that the base nfcore/base image should be tagged to the most recent release. The linting tool compares the tag against the currently installed version.

Additional lines and different metadata can be added without causing the test to fail.

Error #10 - Template TODO statement found

The nf-core workflow template contains a number of comment lines with the following format:

// TODO nf-core: Make some kind of change to the workflow here

This lint test runs through all files in the pipeline and searches for these lines.

Error #11 - Pipeline name

..removed..

Error #12 - Pipeline name

In order to ensure consistent naming, pipeline names should contain only lower case, alphanumeric characters. Otherwise a warning is displayed.

Error #13 - Pipeline name

The nf-core create pipeline template uses cookiecutter behind the scenes. This check fails if any cookiecutter template variables such as {{ cookiecutter.pipeline_name }} are fouund in your pipeline code. Finding a placeholder like this means that something was probably copied and pasted from the template without being properly rendered for your pipeline.

Error #14 - Pipeline schema syntax

Pipelines should have a nextflow_schema.json file that describes the different pipeline parameters (eg. params.something, --something).

For example, an extremely minimal schema could look like this:

{
  "$schema": "https://json-schema.org/draft-07/schema",
  "$id": "https://raw.githubusercontent.com/YOURPIPELINE/master/nextflow_schema.json",
  "title": "YOURPIPELINE pipeline parameters",
  "description": "This pipeline is for testing",
  "properties": {
    "first_param": { "type": "string" }
  },
  "definitions": {
    "my_first_group": {
      "properties": {
        "second_param": { "type": "string" }
      }
    }
  },
  "allOf": [{"$ref": "#/definitions/my_first_group"}]
}

Error #15 - Schema config check

The nextflow_schema.json pipeline schema should describe every flat parameter returned from the nextflow config command (params that are objects or more complex structures are ignored). Missing parameters result in a lint failure.

If any parameters are found in the schema that were not returned from nextflow config a warning is given.