nf-core/eager

There are two possible ways of supplying input sequencing data to nf-core/eager. The most efficient but more simplistic is supplying direct paths (with wildcards) to your FASTQ or BAM files, with each file or pair being considered a single library and each one run independently. TSV input requires creation of an extra file by the user and extra metadata, but allows more powerful lane and library merging.

Direct Input Method

This method is where you specify with --input, the path locations of FASTQ (optionally gzipped) or BAM file(s). This option is mutually exclusive to the TSV input method, which is used for more complex input configurations such as lane and library merging.

When using the direct method of --input you can specify one or multiple samples in one or more directories files. File names must be unique, even if in different directories.

By default, the pipeline assumes you have paired-end data. If you want to run single-end data you must specify --single_end

For example, for a single set of FASTQs, or multiple paired-end FASTQ files in one directory, you can specify:

--input 'path/to/data/sample_*_{1,2}.fastq.gz'

If you have multiple files in different directories, you can use additional wildcards (*) e.g.:

--input 'path/to/data/*/sample_*_{1,2}.fastq.gz'

⚠️ It is not possible to run a mixture of single-end and paired-end files in one run with the paths --input method! Please see the TSV input method for possibilities.

Please note the following requirements:

1. Valid file extensions: .fastq.gz, .fastq, .fq.gz, .fq, .bam.
2. The path must be enclosed in quotes
3. The path must have at least one * wildcard character
4. When using the pipeline with paired end data, the path must use {1,2} notation to specify read pairs.
5. Files names must be unique, having files with the same name, but in different directories is not sufficient
   • This can happen when a library has been sequenced across two sequencers on the same lane. Either rename the file, try a symlink with a unique name, or merge the two FASTQ files prior input.
6. Due to limitations of downstream tools (e.g. FastQC), sample IDs may be truncated after the first . in the name, Ensure file names are unique prior to this!
7. For input BAM files you should provide a small decoy reference genome with pre-made indices, e.g. the human mtDNA or phiX genome, for the mandatory parameter --fasta in order to avoid long computational time for generating the index files of the reference genome, even if you do not actual need a reference genome for any downstream analyses.

TSV Input Method

Alternatively to the direct input method, you can supply to --input a path to a TSV file that contains paths to FASTQ/BAM files and additional metadata. This allows for more complex procedures such as merging of sequencing data across lanes, sequencing runs, sequencing configuration types, and samples.

<p align="center"> <img src="images/usage/merging_files.png" alt="Schematic diagram indicating merging points of different types of libraries, given a TSV input. Dashed boxes are optional library-specific processes" width="70%"> </p>

The use of the TSV --input method is recommended when performing more complex procedures such as lane or library merging. You do not need to specify --single_end, --bam, --colour_chemistry, -udg_type etc. when using TSV input - this is defined within the TSV file itself. You can only supply a single TSV per run (i.e. --input '*.tsv' will not work).

This TSV should look like the following:

A template can be taken from here.

⚠️ Cells must not contain spaces before or after strings, as this will make the TSV unreadable by nextflow. Strings containing spaces should be wrapped in quotes.

When using TSV_input, nf-core/eager will merge FASTQ files of libraries with the same Library_ID but different Lanes values after adapter clipping (and merging), assuming all other metadata columns are the same. If you have the same Library_ID but with different SeqType, this will be merged directly after mapping prior BAM filtering. Finally, it will also merge BAM files with the same Sample_ID but different Library_ID after duplicate removal, but prior to genotyping. Please see caveats to this below.

Column descriptions are as follows:

• Sample_Name: A text string containing the name of a given sample of which there can be multiple libraries. All libraries with the same sample name and same SeqType will be merged after deduplication.
• Library_ID: A text string containing a given library, which there can be multiple sequencing lanes (with the same SeqType).
• Lane: A number indicating which lane the library was sequenced on. Files from the libraries sequenced on different lanes (and different SeqType) will be concatenated after read clipping and merging.
• Colour Chemistry A number indicating whether the Illumina sequencer the library was sequenced on was a 2 (e.g. Next/NovaSeq) or 4 (Hi/MiSeq) colour chemistry machine. This informs whether poly-G trimming (if turned on) should be performed.
• SeqType: A text string of either 'PE' or 'SE', specifying paired end (with both an R1 [or forward] and R2 [or reverse]) and single end data (only R1 [forward], or BAM). This will affect lane merging if different per library.
• Organism: A text string of the organism name of the sample or 'NA'. This currently has no functionality and can be set to 'NA', but will affect lane/library merging if different per library
• Strandedness: A text string indicating whether the library type is 'single' or 'double'. This will affect lane/library merging if different per library.
• UDG_Treatment: A text string indicating whether the library was generated with UDG treatment - either 'full', 'half' or 'none'. Will affect lane/library merging if different per library.
• R1: A text string of a file path pointing to a forward or R1 FASTQ file. This can be used with the R2 column. File names must be unique, even if they are in different directories.
• R2: A text string of a file path pointing to a reverse or R2 FASTQ file, or 'NA' when single end data. This can be used with the R1 column. File names must be unique, even if they are in different directories.
• BAM: A text string of a file path pointing to a BAM file, or 'NA'. Cannot be specified at the same time as R1 or R2, both of which should be set to 'NA'

For example, the following TSV table:

will have the following effects:

• After AdapterRemoval, and prior to mapping, FASTQ files from lane 7 and lane 8 with the same SeqType (and all other metadata columns) will be concatenated together for each Library.
• After mapping, and prior BAM filtering, BAM files with the same with different SeqType (but with all other metadata columns the same) will be merged together for each Library.
• After duplicate removal, BAM files with Library_IDs with the same Sample_Name and the same UDG_Treatment will be merged together.
• If BAM trimming is turned on, all post-trimming BAMs (i.e. non-UDG and half-UDG ) will be merged with UDG-treated (untreated) BAMs, if they have the same Sample_Name.

Note the following important points and limitations for setting up:

• The TSV must use actual tabs (not spaces) between cells.
• File names must be unique regardless of file path, due to risk of over-writing (see: https://github.com/nextflow-io/nextflow/issues/470).
  • If it is 'too late' and you already have duplicate file names, a workaround is to concatenate the FASTQ files together and supply this to a nf-core/eager run. The only downside is that you will not get independent FASTQC results for each file.
• Lane IDs must be unique for each sequencing of each library.
  • If you have a library sequenced e.g. on Lane 8 of two HiSeq runs, you can give a fake lane ID (e.g. 20) for one of the FASTQs, and the libraries will still be processed correctly.
  • This also applies to the SeqType column, i.e. with the example above, if one run is PE and one run is SE, you need to give fake lane IDs to one of the runs as well.
• All BAM files must be specified as SE under SeqType.
  • You should provide a small decoy reference genome with pre-made indices, e.g. the human mtDNA or phiX genome, for the mandatory parameter --fasta in order to avoid long computational time for generating the index files of the reference genome, even if you do not actual need a reference genome for any downstream analyses.
• nf-core/eager will only merge multiple lanes of sequencing runs with the same single-end or paired-end configuration
• Accordingly nf-core/eager will not merge lanes of FASTQs with BAM files (unless you use --run_convertbam), as only FASTQ files are lane-merged together.
• Same libraries that are sequenced on different sequencing configurations (i.e single- and paired-end data), will be merged after mapping and will always be considered 'paired-end' during downstream processes
  • Important running DeDup in this context is not recommended, as PE and SE data at the same position will not be evaluated as duplicates. Therefore not all duplicates will be removed.
  • When you wish to run PE/SE data together -dedupper markduplicates is therefore preferred.
  • An error will be thrown if you try to merge both PE and SE and also supply --skip_merging.
  • If you truly want to mix SE data and PE data but using mate-pair info for PE mapping, please run FASTQ preprocessing mapping manually and supply BAM files for downstream processing by nf-core/eager
  • If you regularly want to run the situation above, please leave a feature request on github.
• DamageProfiler, NuclearContamination, MTtoNucRatio and PreSeq are performed on each unique library separately after deduplication (but prior same-treated library merging).
• nf-core/eager functionality such as --run_trim_bam will be applied to only non-UDG (UDG_Treatment: none) or half-UDG (UDG_Treatment: half) libraries.
• Qualimap is run on each sample, after merging of libraries (i.e. your values will reflect the values of all libraries combined - after being damage trimmed etc.).
• Genotyping will be typically performed on each sample independently, as normally all libraries will have been merged together. However, if you have a mixture of single-stranded and double-stranded libraries, you will normally need to genotype separately. In this case you must give each the SS and DS libraries distinct Sample_IDs; otherwise you will receive a file collision error in steps such as sexdeterrmine, and then you will need to merge these yourself. We will consider changing this behaviour in the future if there is enough interest.

Defines whether Uracil-DNA glycosylase (UDG) treatment was used to remove DNA damage on the sequencing libraries.

Specify 'none' if no treatment was performed. If you have partial UDG treated data (Rohland et al 2016), specify 'half'. If you have complete UDG treated data (Briggs et al. 2010), specify 'full'.

When also using PMDtools specifying 'half' will use a different model for DNA damage assessment in PMDTools (PMDtools: --UDGhalf). Specify 'full' and the PMDtools DNA damage assessment will use CpG context only (PMDtools: --CpG). Default: 'none'.

Tip: You should provide a small decoy reference genome with pre-made indices, e.g. the human mtDNA genome, for the mandatory parameter --fasta in order to avoid long computational time for generating the index files of the reference genome, even if you do not actual need a reference genome for any downstream analyses.

Indicates libraries are single stranded.

Currently only affects MALTExtract where it will switch on damage patterns calculation mode to single-stranded, (MaltExtract: --singleStranded) and genotyping with pileupCaller where a different method is used (pileupCaller: --singleStrandMode). Default: false

Only required when using the 'Path' method of --input

By default, the pipeline expects paired-end data. If you have single-end data, specify this parameter on the command line when you launch the pipeline. It is not possible to run a mixture of single-end and paired-end files in one run.

Only required when using the 'Path' method of --input

Specifies which Illumina colour chemistry a library was sequenced with. This informs whether to perform poly-G trimming (if --complexity_filter_poly_g is also supplied). Only 2 colour chemistry sequencers (e.g. NextSeq or NovaSeq) can generate uncertain poly-G tails (due to 'G' being indicated via a no-colour detection). Default is '4' to indicate e.g. HiSeq or MiSeq platforms, which do not require poly-G trimming. Options: 2, 4. Default: 4

Only required when using the 'Path' method of input.

Specifies the input file type to --input is in BAM format. This will automatically also apply --single_end.

Only required when using the 'Path' method of --input.

Can be used to set a path to a BED file (3/6 column format) of SNP positions of a reference genome, to calculate SNP captured libraries on-target efficiency. This should be used for array or in-solution SNP capture protocols such as 390K, 1240K, etc. If supplied, on-target metrics are automatically generated for you by qualimap.

Allows you to convert an input BAM file back to FASTQ for downstream processing. Note this is required if you need to perform AdapterRemoval and/or polyG clipping.

If not turned on, BAMs will automatically be sent to post-mapping steps.

You specify the full path to your reference genome here. The FASTA file can have any file suffix, such as .fasta, .fna, .fa, .FastA etc. You may also supply a gzipped reference files, which will be unzipped automatically for you.

For example:

--fasta '/<path>/<to>/my_reference.fasta'

If you don't specify appropriate --bwa_index, --fasta_index parameters, the pipeline will create these indices for you automatically. Note that you can save the indices created for you for later by giving the --save_reference flag. You must select either a --fasta or --genome

Alternatively to --fasta, the pipeline config files come bundled with paths to the Illumina iGenomes reference index files. If running with docker or AWS, the configuration is set up to use the AWS-iGenomes resource.

There are 31 different species supported in the iGenomes references. To run the pipeline, you must specify which to use with the --genome flag.

You can find the keys to specify the genomes in the iGenomes config file. Common genomes that are supported are:

• Human
  • --genome GRCh37
  • --genome GRCh38
• Mouse *
  • --genome GRCm38
• Drosophila *
  • --genome BDGP6
• S. cerevisiae *
  • --genome 'R64-1-1'

* Not bundled with nf-core eager by default.

Note that you can use the same configuration setup to save sets of reference files for your own use, even if they are not part of the iGenomes resource. See the Nextflow documentation for instructions on where to save such a file.

The syntax for this reference configuration is as follows:

params {
  genomes {
    'GRCh37' {
      fasta   = '<path to the iGenomes genome fasta file>'
    }
    // Any number of additional genomes, key is used with --genome
  }
}

If you want to use pre-existing bwa index indices, please supply the directory to the FASTA you also specified in --fasta nf-core/eager will automagically detect the index files by searching for the FASTA filename with the corresponding bwa index file suffixes.

For example:

nextflow run nf-core/eager \
-profile test,docker \
--input '*{R1,R2}*.fq.gz'
--fasta 'results/reference_genome/bwa_index/BWAIndex/Mammoth_MT_Krause.fasta' \
--bwa_index 'results/reference_genome/bwa_index/BWAIndex/'

bwa index does not give you an option to supply alternative suffixes/names for these indices. Thus, the file names generated by this command must not be changed, otherwise nf-core/eager will not be able to find them.

If you want to use pre-existing bt2 index indices, please supply the directory to the FASTA you also specified in --fasta. nf-core/eager will automagically detect the index files by searching for the FASTA filename with the corresponding bt2 index file suffixes.

For example:

nextflow run nf-core/eager \
-profile test,docker \
--input '*{R1,R2}*.fq.gz'
--fasta 'results/reference_genome/bwa_index/BWAIndex/Mammoth_MT_Krause.fasta' \
--bwa_index 'results/reference_genome/bt2_index/BT2Index/'

bowtie2-build does not give you an option to supply alternative suffixes/names for these indices. Thus, the file names generated by this command must not be changed, otherwise nf-core/eager will not be able to find them.

If you want to use a pre-existing samtools faidx index, use this to specify the required FASTA index file for the selected reference genome. This should be generated by samtools faidx and has a file suffix of .fai

For example:

--fasta_index 'Mammoth_MT_Krause.fasta.fai'

If you want to use a pre-existing picard CreateSequenceDictionary dictionary file, use this to specify the required .dict file for the selected reference genome.

For example:

--seq_dict 'Mammoth_MT_Krause.dict'

This parameter is required to be set for large reference genomes. If your reference genome is larger than 3.5GB, the samtools index calls in the pipeline need to generate CSI indices instead of BAI indices to compensate for the size of the reference genome (with samtools: -c). This parameter is not required for smaller references (including the human hg19 or grch37/grch38 references), but >4GB genomes have been shown to need CSI indices. Default: off

Use this if you do not have pre-made reference FASTA indices for bwa, samtools and picard. If you turn this on, the indices nf-core/eager generates for you and will be saved in the <your_output_dir>/results/reference_genomes for you. If not supplied, nf-core/eager generated index references will be deleted.

modifies SAMtools index command: -c

The output directory where the results will be saved. By default will be made in the directory you run the command in under ./results.

Nextflow mode for 'publishing' final results files i.e. how to move final files into your --outdir from working directories. Options: 'symlink', 'rellink', 'link', 'copy', 'copyNoFollow', 'move'. Default: 'copy'.

It is recommended to select copy (default) if you plan to regularly delete intermediate files from work/.

An email address to send a summary email to when the pipeline is completed.

Turns off FastQC pre- and post-Adapter Removal, to speed up the pipeline. Use of this flag is most common when data has been previously pre-processed and the post-Adapter Removal mapped reads are being re-mapped to a new reference genome.

Turns off adapter trimming and paired-end read merging. Equivalent to setting both --skip_collapse and --skip_trim.

Turns off the computation of library complexity estimation.

Turns off duplicate removal methods DeDup and MarkDuplicates respectively. No duplicates will be removed on any data in the pipeline.

Turns off the DamageProfiler module to compute DNA damage profiles.

Turns off QualiMap and thus does not compute coverage and other mapping metrics.

Performs a poly-G tail removal step in the beginning of the pipeline using fastp, if turned on. This can be useful for trimming ploy-G tails from short-fragments sequenced on two-colour Illumina chemistry such as NextSeqs (where no-fluorescence is read as a G on two-colour chemistry), which can inflate reported GC content values.

This option can be used to define the minimum length of a poly-G tail to begin low complexity trimming. By default, this is set to a value of 10 unless the user has chosen something specifically using this option.

Modifies fastp parameter: --poly_g_min_len

Defines the adapter sequence to be used for the forward read. By default, this is set to 'AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC'.

Modifies AdapterRemoval parameter: --adapter1

Defines the adapter sequence to be used for the reverse read in paired end sequencing projects. This is set to 'AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTA' by default.

Modifies AdapterRemoval parameter: --adapter2

Defines the minimum read length that is required for reads after merging to be considered for downstream analysis after read merging. Default is 30.

Note that performing read length filtering at this step is not reliable for correct endogenous DNA calculation, when you have a large percentage of very short reads in your library - such as retrieved in single-stranded library protocols. When you have very few reads passing this length filter, it will artificially inflate your endogenous DNA by creating a very small denominator. In these cases it is recommended to set this to 0, and use --bam_filter_minreadlength instead, to filter out 'un-usable' short reads after mapping.

Modifies AdapterRemoval parameter: --minlength

Defines the minimum read quality per base that is required for a base to be kept. Individual bases at the ends of reads falling below this threshold will be clipped off. Default is set to 20.

Modifies AdapterRemoval parameter: --minquality

Sets the minimum overlap between two reads when read merging is performed. Default is set to 1 base overlap.

Modifies AdapterRemoval parameter: --minadapteroverlap

Turns off the paired-end read merging.

For example

--skip_collapse  --input '*_{R1,R2}_*.fastq'

It is important to use the paired-end wildcard globbing as --skip_collapse can only be used on paired-end data!

⚠️ If you run this and also with --clip_readlength set to something (as is by default), you may end up removing single reads from either the pair1 or pair2 file. These will be NOT be mapped when aligning with either bwa or bowtie, as both can only accept one (forward) or two (forward and reverse) FASTQs as input.

Modifies AdapterRemoval parameter: --collapse

Turns off adapter AND quality trimming.

For example:

--skip_trim  --input '*.fastq'

⚠️ it is not possible to keep quality trimming (n or base quality) on, and skip adapter trimming.

⚠️ it is not possible to turn off one or the other of quality trimming or n trimming. i.e. --trimns --trimqualities are both given or neither. However setting quality in --clip_min_read_quality to 0 would theoretically turn off base quality trimming.

Modifies AdapterRemoval parameters: --trimns --trimqualities --adapter1 --adapter2

Turns off quality based trimming at the 5p end of reads when any of the --trimns, --trimqualities, or --trimwindows options are used. Only 3p end of reads will be removed.

This also entirely disables quality based trimming of collapsed reads, since both ends of these are informative for PCR duplicate filtering. Described here.

Modifies AdapterRemoval parameters: --preserve5p

Specify that only merged reads are sent downstream for analysis.

Singletons (i.e. reads missing a pair), or un-merged reads (where there wasn't sufficient overlap) are discarded.

You may want to use this if you want ensure only the best quality reads for your analysis, but with the penalty of potentially losing still valid data (even if some reads have slightly lower quality). It is highly recommended when using --dedupper 'dedup' (see below).

Specify which mapping tool to use. Options are BWA aln ('bwaaln'), BWA mem ('bwamem'), circularmapper ('circularmapper'), or bowtie2 (bowtie2). BWA aln is the default and highly suited for short-read ancient DNA. BWA mem can be quite useful for modern DNA, but is rarely used in projects for ancient DNA. CircularMapper enhances the mapping procedure to circular references, using the BWA algorithm but utilizing a extend-remap procedure (see Peltzer et al 2016, Genome Biology for details). Bowtie2 is similar to BWA aln, and has recently been suggested to provide slightly better results under certain conditions (Poullet and Orlando 2020), as well as providing extra functionality (such as FASTQ trimming). Default is 'bwaaln'

More documentation can be seen for each tool under:

• BWA aln
• BWA mem
• CircularMapper
• Bowtie2

Configures the bwa aln -n parameter, defining how many mismatches are allowed in a read. By default set to 0.04 (following recommendations of Schubert et al. (2012 BMC Genomics)), if you're uncertain what to set check out this Shiny App for more information on how to set this parameter efficiently.

Modifies bwa aln parameter: -n

Configures the bwa aln -k parameter for the seeding phase in the mapping algorithm. Default is set to 2.

Modifies BWA aln parameter: -k

Configures the length of the seed used in bwa aln -l. Default is set to be 'turned off' at the recommendation of Schubert et al. (2012 BMC Genomics) for ancient DNA with 1024.

Note: Despite being recommended, turning off seeding can result in long runtimes!

Modifies BWA aln parameter: -l

The number of bases to extend the reference genome with. By default this is set to 500 if not specified otherwise.

Modifies circulargenerator and realignsamfile parameter: -e

The chromosome in your FASTA reference that you'd like to be treated as circular. By default this is set to MT but can be configured to match any other chromosome.

Modifies circulargenerator parameter: -s

If you want to filter out reads that don't map to a circular chromosome, turn this on. By default this option is turned off.

The type of read alignment to use. Options are 'local' or 'end-to-end'. Local allows only partial alignment of read, with ends of reads possibly 'soft-clipped' (i.e. remain unaligned/ignored), if the soft-clipped alignment provides best alignment score. End-to-end requires all nucleotides to be aligned. Default is 'local', following Cahill et al (2018) and Poullet and Orlando 2020.

Modifies Bowtie2 parameters: --very-fast --fast --sensitive --very-sensitive --very-fast-local --fast-local --sensitive-local --very-sensitive-local

The Bowtie2 'preset' to use. Options: 'no-preset' 'very-fast', 'fast', 'sensitive', or 'very-sensitive'. These strings apply to both --bt2_alignmode options. See the Bowtie2 manual for actual settings. Default is 'sensitive' (following Poullet and Orlando (2020), when running damaged-data without UDG treatment)

Modifies Bowtie2 parameters: --very-fast --fast --sensitive --very-sensitive --very-fast-local --fast-local --sensitive-local --very-sensitive-local

The number of mismatches allowed in the seed during seed-and-extend procedure of Bowtie2. This will override any values set with --bt2_sensitivity. Can either be 0 or 1. Default: 0 (i.e. use--bt2_sensitivity defaults).

Modifies Bowtie2 parameters: -N

The length of the seed sub-string to use during seeding. This will override any values set with --bt2_sensitivity. Default: 0 (i.e. use--bt2_sensitivity defaults: 20 for local and 22 for end-to-end.

Modifies Bowtie2 parameters: -L

Number of bases to trim at the 5' (left) end of read prior alignment. Maybe useful when left-over sequencing artefacts of in-line barcodes present Default: 0

Modifies Bowtie2 parameters: -bt2_trim5

Number of bases to trim at the 3' (right) end of read prior alignment. Maybe useful when left-over sequencing artefacts of in-line barcodes present Default: 0.

Modifies Bowtie2 parameters: -bt2_trim3

Create pre-Adapter Removal FASTQ files without reads that mapped to reference (e.g. for public upload of privacy sensitive non-host data)

Read removal mode. Remove mapped reads completely ('remove') or just replace mapped reads sequence by N ('replace')

Modifies extract_map_reads.py parameter: -m

Turns on the bam filtering module for either mapping quality filtering or unmapped read treatment.

Specify a mapping quality threshold for mapped reads to be kept for downstream analysis. By default keeps all reads and is therefore set to 0 (basically doesn't filter anything).

Modifies samtools view parameter: -q

Specify minimum length of mapped reads. This filtering will apply at the same time as mapping quality filtering.

If used instead of minimum length read filtering at AdapterRemoval, this can be useful to get more realistic endogenous DNA percentages, when most of your reads are very short (e.g. in single-stranded libraries) and would otherwise be discarded by AdapterRemoval (thus making an artificially small denominator for a typical endogenous DNA calculation). Note in this context you should not perform mapping quality filtering nor discarding of unmapped reads to ensure a correct denominator of all reads, for the endogenous DNA calculation.

Modifies filter_bam_fragment_length.py parameter: -l

Defines how to proceed with unmapped reads: 'discard' removes all unmapped reads, keep keeps both unmapped and mapped reads in the same BAM file, 'bam' keeps unmapped reads as BAM file, 'fastq' keeps unmapped reads as FastQ file, both keeps both BAM and FASTQ files. Default is discard. keep is what would happen if --run_bam_filtering was not supplied.

Note that in all cases, if --bam_mapping_quality_threshold is also supplied, mapping quality filtering will still occur on the mapped reads.

Modifies samtools view parameter: -f4 -F4

Sets the duplicate read removal tool. By default uses markduplicates from Picard. Alternatively an ancient DNA specific read deduplication tool dedup (Peltzer et al. 2016) is offered.

This utilises both ends of paired-end data to remove duplicates (i.e. true exact duplicates, as markduplicates will over-zealously deduplicate anything with the same starting position even if the ends are different). DeDup should only be used solely on paired-end data otherwise suboptimal deduplication can occur if applied to either single-end or a mix of single-end/paired-end data.

Note that if you run without the --mergedonly flag for AdapterRemoval, DeDup will likely fail. If you absolutely want to use both PE and SE data, you can supply the --dedup_all_merged flag to consider singletons to also be merged paired-end reads. This may result in over-zealous deduplication.

Sets DeDup to treat all reads as merged reads. This is useful if reads are for example not prefixed with M_ in all cases. Therefore, this can be used as a workaround when also using a mixture of paired-end and single-end data, however this is not recommended (see above).

Modifies dedup parameter: -m

Can be used to configure the step size of Preseq's c_curve method. Can be useful when only few and thus shallow sequencing results are used for extrapolation.

Modifies preseq c_curve parameter: -s

Specifies the length filter for DamageProfiler. By default set to 100.

Modifies DamageProfile parameter: -l

Specifies the length of the read start and end to be considered for profile generation in DamageProfiler. By default set to 15 bases.

Modifies DamageProfile parameter: -t

Specifies what the maximum misincorporation frequency should be displayed as, in the DamageProfiler damage plot. This is set to 0.30 (i.e. 30%) by default as this matches the popular mapDamage2.0 program. However, the default behaviour of DamageProfiler is to 'autoscale' the y-axis maximum to zoom in on any possible damage that may occur (e.g. if the damage is about 10%, the highest value on the y-axis would be set to 0.12). This 'autoscale' behaviour can be turned on by specifying the number to 0. Default: 0.30.

Modifies DamageProfile parameter: -yaxis_damageplot

Specifies to run PMDTools for damage based read filtering and assessment of DNA damage in sequencing libraries. By default turned off.

Specifies the range in which to consider DNA damage from the ends of reads. By default set to 10.

Modifies PMDTools parameter: --range

Specifies the PMDScore threshold to use in the pipeline when filtering BAM files for DNA damage. Only reads which surpass this damage score are considered for downstream DNA analysis. By default set to 3 if not set specifically by the user.

Modifies PMDTools parameter: --threshold

Can be used to set a path to a reference genome mask for PMDTools.

The maximum number of reads used for damage assessment in PMDtools. Can be used to significantly reduce the amount of time required for damage assessment in PMDTools. Note that a too low value can also obtain incorrect results.

Modifies PMDTools parameter: -n

Specifies to turn on the bedtools module, producing statistics for breadth (or percent coverage), and depth (or X fold) coverages.

Specify the path to a GFF/BED containing the feature coordinates (or any acceptable input for bedtools coverage). Must be in quotes.

Turns on the BAM trimming method. Trims off [n] bases from reads in the deduplicated BAM file. Damage assessment in PMDTools or DamageProfiler remains untouched, as data is routed through this independently. BAM trimming is typically performed to reduce errors during genotyping that can be caused by aDNA damage.

BAM trimming will only be performed on libraries indicated as --udg_type 'none' or --udg_type 'half'. Complete UDG treatment ('full') should have removed all damage. The amount of bases that will be trimmed off can be set separately for libraries with --udg_type 'none' and 'half' (see --bamutils_clip_half_udg_left / --bamutils_clip_half_udg_right / --bamutils_clip_none_udg_left / --bamutils_clip_none_udg_right).

Note: additional artefacts such as bar-codes or adapters that could potentially also be trimmed should be removed prior mapping.

Default set to 1 and clips off one base of the left or right side of reads from libraries whose UDG treatment is set to half. Note that reverse reads will automatically be clipped off at the reverse side with this (automatically reverses left and right for the reverse read).

Modifies bamUtil's trimBam parameter: -L -R

Default set to 1 and clips off one base of the left or right side of reads from libraries whose UDG treatment is set to half. Note that reverse reads will automatically be clipped off at the reverse side with this (automatically reverses left and right for the reverse read).

Modifies bamUtil's trimBam parameter: -L -R

Default set to 1 and clips off one base of the left or right side of reads from libraries whose UDG treatment is set to none. Note that reverse reads will automatically be clipped off at the reverse side with this (automatically reverses left and right for the reverse read).

Modifies bamUtil's trimBam parameter: -L -R

Default set to 1 and clips off one base of the left or right side of reads from libraries whose UDG treatment is set to none. Note that reverse reads will automatically be clipped off at the reverse side with this (automatically reverses left and right for the reverse read).

Modifies bamUtil's trimBam parameter: -L -R

By default, nf-core/eager uses hard clipping and sets clipped bases to N with quality ! in the BAM output. Turn this on to use soft-clipping instead, masking reads at the read ends respectively using the CIGAR string.

Modifies bam trimBam parameter: -c

Turns on genotyping to run on all post-dedup and downstream BAMs. For example if --run_pmdtools and --trim_bam are both supplied, the genotyper will be run on all three BAM files i.e. post-deduplication, post-pmd and post-trimmed BAM files.

Specifies which genotyper to use. Current options are: GATK (v3.5) UnifiedGenotyper or GATK Haplotype Caller (v4); and the FreeBayes Caller. Specify 'ug', 'hc', 'freebayes', 'pileupcaller' and 'angsd' respectively.

Note that while UnifiedGenotyper is more suitable for low-coverage ancient DNA (HaplotypeCaller does de novo assembly around each variant site), it is officially deprecated by the Broad Institute and is only accessible by an archived version not properly available on conda. Therefore if specifying 'ug', will need to supply a GATK 3.5 -jar to the parameter gatk_ug_jar. Note that this means the pipline is not fully reproducible in this configuration, unless you personally supply the .jar file.

Indicates which BAM file to use for genotyping, depending on what BAM processing modules you have turned on. Options are: 'raw' for mapped only, filtered, or DeDup BAMs (with priority right to left); 'trimmed' (for base clipped BAMs); 'pmd' (for pmdtools output). Default is: 'raw'.

Specify a path to a local copy of a GATK 3.5 .jar file, preferably version '3.5-0-g36282e4'. The download location of this may be available from the GATK forums or the Google Cloud Storage of the Broad Institute.

If selected, specify a GATK genotyper phred-scaled confidence threshold of a given SNP/INDEL call. Default: 30

Modifies GATK UnifiedGenotyper or HaplotypeCaller parameter: -stand_call_conf

If selected, specify a GATK genotyper ploidy value of your reference organism. E.g. if you want to allow heterozygous calls from >= diploid organisms. Default: 2

Modifies GATK UnifiedGenotyper or HaplotypeCaller parameter: --sample-ploidy

Maximum depth coverage allowed for genotyping before down-sampling is turned on. Any position with a coverage higher than this value will be randomly down-sampled to 250 reads. Default: 250

Modifies GATK UnifiedGenotyper parameter: -dcov

(Optional) Specify VCF file for output VCF SNP annotation e.g. if you want to annotate your VCF file with 'rs' SNP IDs. Check GATK documentation for more information. Gzip not accepted.

If the GATK genotyper HaplotypeCaller is selected, what type of VCF to create, i.e. produce calls for every site or just confidence sites. Options: 'EMIT_VARIANTS_ONLY', 'EMIT_ALL_CONFIDENT_SITES', 'EMIT_ALL_ACTIVE_SITES'. Default: 'EMIT_VARIANTS_ONLY'

Modifies GATK HaplotypeCaller parameter: -output_mode

If the GATK HaplotypeCaller is selected, mode for emitting reference confidence calls. Options: 'NONE', 'BP_RESOLUTION', 'GVCF'. Default: 'GVCF'

Modifies GATK HaplotypeCaller parameter: --emit-ref-confidence

If the GATK UnifiedGenotyper is selected, what type of VCF to create, i.e. produce calls for every site or just confidence sites. Options: 'EMIT_VARIANTS_ONLY', 'EMIT_ALL_CONFIDENT_SITES', 'EMIT_ALL_SITES'. Default: 'EMIT_VARIANTS_ONLY'

Modifies GATK UnifiedGenotyper parameter: --output_mode

If the GATK UnifiedGenotyper is selected, which likelihood model to follow, i.e. whether to call use SNPs or INDELS etc. Options: 'SNP', 'INDEL', 'BOTH', 'GENERALPLOIDYSNP', 'GENERALPLOIDYINDEL'. Default: 'SNP'

Modifies GATK UnifiedGenotyper parameter: --genotype_likelihoods_model

If provided when running GATK's UnifiedGenotyper, this will put into the output folder the BAMs that have realigned reads (with GATK's (v3) IndelRealigner) around possible variants for improved genotyping.

These BAMs will be stored in the same folder as the corresponding VCF files.

When running GATK's UnifiedGenotyper, specify a value to set base quality scores, if reads are missing this information. Might be useful if you have 'synthetically' generated reads (e.g. chopping up a reference genome). Default is set to -1 which is to not set any default quality (turned off). Default: -1

Modifies GATK UnifiedGenotyper parameter: --defaultBaseQualities

Specify minimum required supporting observations to consider a variant. Default: 1

Modifies freebayes parameter: -C

Specify to skip over regions of high depth by discarding alignments overlapping positions where total read depth is greater than specified C. Not set by default.

Modifies freebayes parameter: -g

Specify ploidy of sample in FreeBayes. Default is diploid. Default: 2

Modifies freebayes parameter: -p

Specify a SNP panel in the form of a bed file of sites at which to generate pileup for pileupCaller.

Specify a SNP panel in EIGENSTRAT format, pileupCaller will call these sites.

Specify calling method to use. Options: randomHaploid, randomDiploid, majorityCall. Default: 'randomHaploid'

Modifies pileupCaller parameter: --randomHaploid --randomDiploid --majorityCall

Specify if genotypes of transition SNPs should be called, set to missing, or excluded from the genotypes respectively. Options: 'AllSites', 'TransitionsMissing', 'SkipTransitions'. Default: 'AllSites'

Modifies pileupCaller parameter: --skipTransitions --transitionsMissing

Specify which genotype likelihood model to use. Options: 'samtools, 'gatk', 'soapsnp', 'syk'. Default: 'samtools'

Modifies ANGSD parameter: -GL

Specifies what type of genotyping likelihood file format will be output. Options: 'text', 'binary', 'binary_three', 'beagle_binary'. Default: 'text'.

The options refer to the following descriptions respectively:

• text: textoutput of all 10 log genotype likelihoods.
• binary: binary all 10 log genotype likelihood
• binary_three: binary 3 times likelihood
• beagle_binary: beagle likelihood file

See the ANGSD documentation for more information on which to select for your downstream applications.

Modifies ANGSD parameter: -doGlF

Turns on the ANGSD creation of a FASTA file from the BAM file.

The type of base calling to be performed when creating the ANGSD FASTA file. Options: 'random' or 'common'. Will output the most common non-N base at each given position, whereas 'random' will pick one at random. Default: 'random'.

Modifies ANGSD parameter: -doFasta -doCounts

Turn on consensus sequence genome creation via VCF2Genome. Only accepts GATK UnifiedGenotyper VCF files with the --gatk_ug_out_mode 'EMIT_ALL_SITES' and --gatk_ug_genotype_model 'SNP flags. Typically useful for small genomes such as mitochondria.

The name of your requested output FASTA file. Do not include .fasta suffix.

The name of the FASTA entry you would like in your FASTA file.

Minimum depth coverage for a SNP to be made. Else, a SNP will be called as N. Default: 5

Modifies VCF2Genome parameter: -minc

Minimum genotyping quality of a call to be made. Else N will be called. Default: 30

Modifies VCF2Genome parameter: -minq

In the case of two possible alleles, the frequency of the majority allele required for a call to be made. Else, a SNP will be called as N. Default: 0.8

Modifies VCF2Genome parameter: -minfreq

Turns on MultiVCFAnalyzer. Will only work when in combination with UnifiedGenotyper genotyping module.

Specify whether to tell MultiVCFAnalyzer to write within the SNP table the frequencies of the allele at that position e.g. A (70%).

The minimal genotyping quality for a SNP to be considered for processing by MultiVCFAnalyzer. The default threshold is 30.

The minimal number of reads covering a base for a SNP at that position to be considered for processing by MultiVCFAnalyzer. The default depth is 5.

The minimal frequency of a nucleotide for a 'homozygous' SNP to be called. In other words, e.g. 90% of the reads covering that position must have that SNP to be called. If the threshold is not reached, and the previous two parameters are matched, a reference call is made (displayed as . in the SNP table). If the above two parameters are not met, an 'N' is called. The default allele frequency is 0.9.

The minimum frequency of a nucleotide for a 'heterozygous' SNP to be called. If this parameter is set to the same as --min_allele_freq_hom, then only homozygous calls are made. If this value is less than the previous parameter, then a SNP call will be made. If it is between this and the previous parameter, it will be displayed as a IUPAC uncertainty call. Default is 0.9.

If you wish to add to the table previously created VCF files, specify here a path with wildcards (in quotes). These VCF files must be created the same way as your settings for GATK UnifiedGenotyping module above.

If you wish to report in the SNP table annotation information for the regions SNPs fall in, provide a file in GFF format (the path must be in quotes).

If you wish to exclude SNP regions from consideration by MultiVCFAnalyzer (such as for problematic regions), provide a file in GFF format (the path must be in quotes).

If you wish to include results from SNPEff effect analysis, supply the output from SNPEff in txt format (the path must be in quotes).

Turn on the module to estimate the ratio of mitochondrial to nuclear reads.

Specify the FASTA entry in the reference file specified as --fasta, which acts as the mitochondrial 'chromosome' to base the ratio calculation on. The tool only accepts the first section of the header before the first space. The default chromosome name is based on hs37d5/GrCH37 human reference genome. Default: 'MT'

Specify to run the optional process of sex determination.

Specify an optional bedfile of the list of SNPs to be used for X-/Y-rate calculation. Running without this parameter will considerably increase runtime, and render the resulting error bars untrustworthy. Theoretically, any set of SNPs that are distant enough that two SNPs are unlikely to be covered by the same read can be used here. The programme was coded with the 1240K panel in mind. The path must be in quotes.

Specify to run the optional processes for (human) nuclear DNA contamination estimation.

The name of the human chromosome X in your bam. 'X' for hs37d5, 'chrX' for HG19. Defaults to 'X'.

Turn on the metagenomic screening module.

Specify which taxonomic classifier to use. There are two options available:

• kraken for Kraken2
• malt for MALT

⚠️ Important It is very important to run nextflow clean -f on your Nextflow run directory once completed. RMA6 files are VERY large and are copied from a work/ directory into the results folder. You should clean the work directory with the command to ensure non-redundancy and large HDD footprints!

Specify the path to the directory containing your taxonomic classifier's database (malt or kraken).

For Kraken2, it can be either the path to the directory or the path to the .tar.gz compressed directory of the Kraken2 database.

Specify the minimum number of reads a given taxon is required to have to be retained as a positive 'hit'.
For malt, this only applies when --malt_min_support_mode is set to 'reads'. Default: 1.

Modifies MALT or kraken_parse.py parameter: -sup and -c respectively

Specify the minimum percent identity (or similarity) a sequence must have to the reference for it to be retained. Default is 85

Only used when --metagenomic_tool malt is also supplied.

Modifies MALT parameter: -id

Use this to run the program in 'BlastN', 'BlastP', 'BlastX' modes to align DNA and DNA, protein and protein, or DNA reads against protein references respectively. Ensure your database matches the mode. Check the MALT manual for more details. Default: 'BlastN'

Only when --metagenomic_tool malt is also supplied.

Modifies MALT parameter: -m

Specify what alignment algorithm to use. Options are 'Local' or 'SemiGlobal'. Local is a BLAST like alignment, but is much slower. Semi-global alignment aligns reads end-to-end. Default: 'SemiGlobal'

Only when --metagenomic_tool malt is also supplied.

Modifies MALT parameter: -at

Specify the top percent value of the LCA algorithm. From the MALT manual: "For each read, only those matches are used for taxonomic placement whose bit disjointScore is within 10% of the best disjointScore for that read.". Default: 1.

Only when --metagenomic_tool malt is also supplied.

Modifies MALT parameter: -top

Specify whether to use a percentage, or raw number of reads as the value used to decide the minimum support a taxon requires to be retained.

Only when --metagenomic_tool malt is also supplied.

Modifies MALT parameter: -sup -supp

Specify the minimum number of reads (as a percentage of all assigned reads) a given taxon is required to have to be retained as a positive 'hit' in the RMA6 file. This only applies when --malt_min_support_mode is set to 'percent'. Default 0.01.

Only when --metagenomic_tool malt is also supplied.

Modifies MALT parameter: -supp

Specify the maximum number of alignments a read can have. All further alignments are discarded. Default: 100

Only when --metagenomic_tool malt is also supplied.

Modifies MALT parameter: -mq

How to load the database into memory. Options are 'load', 'page' or 'map'. 'load' directly loads the entire database into memory prior seed look up, this is slow but compatible with all servers/file systems. 'page' and 'map' perform a sort of 'chunked' database loading, allowing seed look up prior entire database loading. Note that Page and Map modes do not work properly not with many remote file-systems such as GPFS. Default is 'load'.

Only when --metagenomic_tool malt is also supplied.

Modifies MALT parameter: --memoryMode

Specify to also produce gzipped SAM files of all alignments and un-aligned reads in addition to RMA6 files. These are not soft-clipped or in 'sparse' format. Can be useful for downstream analyses due to more common file format.

⚠️ can result in very large run output directories as this is essentially duplication of the RMA6 files.

Modifies MALT parameter -a -f

Turn on MaltExtract for MALT aDNA characteristics authentication of metagenomic output from MALT.

More can be seen in the MaltExtract documentation

Only when --metagenomic_tool malt is also supplied

Path to a .txt file with taxa of interest you wish to assess for aDNA characteristics. In .txt file should be one taxon per row, and the taxon should be in a valid NCBI taxonomy name format.

Only when --metagenomic_tool malt is also supplied.

Path to directory containing containing the NCBI resource tree and taxonomy table files (ncbi.tre and ncbi.map; available at the HOPS repository).

Only when --metagenomic_tool malt is also supplied.

Specify which MaltExtract filter to use. This is used to specify what types of characteristics to scan for. The default will output statistics on all alignments, and then a second set with just reads with one C to T mismatch in the first 5 bases. Further details on other parameters can be seen in the HOPS documentation. Options: 'def_anc', 'ancient', 'default', 'crawl', 'scan', 'srna', 'assignment'. Default: 'def_anc'.

Only when --metagenomic_tool malt is also supplied.

Modifies MaltExtract parameter: -f

Specify frequency of top alignments for each read to be considered for each node. Default is 0.01, i.e. 1% of all reads (where 1 would correspond to 100%).

⚠️ this parameter follows the same concept as --malt_top_percent but uses a different notation i.e. integer (MALT) versus float (MALTExtract)

Default: 0.01.

Only when --metagenomic_tool malt is also supplied.

Modifies MaltExtract parameter: -a

Turn off destacking. If left on, a read that overlaps with another read will be removed (leaving a depth coverage of 1).

Only when --metagenomic_tool malt is also supplied.

Modifies MaltExtract parameter: --destackingOff

Turn off downsampling. By default, downsampling is on and will randomly select 10,000 reads if the number of reads on a node exceeds this number. This is to speed up processing, under the assumption at 10,000 reads the species is a 'true positive'.

Only when --metagenomic_tool malt is also supplied.

Modifies MaltExtract parameter: --downSampOff

Turn off duplicate removal. By default, reads that are an exact copy (i.e. same start, stop coordinate and exact sequence match) will be removed as it is considered a PCR duplicate.

Only when --metagenomic_tool malt is also supplied.

Modifies MaltExtract parameter: --dupRemOff

Export alignments of hits for each node in BLAST format. By default turned off.

Only when --metagenomic_tool malt is also supplied.

Modifies MaltExtract parameter: --matches

Export 'minimal' summary files (i.e. without alignments) that can be loaded into MEGAN6. By default turned off.

Only when --metagenomic_tool malt is also supplied.

Modifies MaltExtract parameter: --meganSummary

Minimum percent identity alignments are required to have to be reported. Higher values allows fewer mismatches between read and reference sequence, but therefore will provide greater confidence in the hit. Lower values allow more mismatches, which can account for damage and divergence of a related strain/species to the reference. Recommended to set same as MALT parameter or higher. Default: 85.0.

Only when --metagenomic_tool malt is also supplied.

Modifies MaltExtract parameter: --minPI

Use the best alignment of each read for every statistic, except for those concerning read distribution and coverage. Default: off.

Only when --metagenomic_tool malt is also supplied.

Modifies MaltExtract parameter: --useTopAlignment