Pipeline Updates

• 🔄 Updated template to nf-core/tools v2.9
• 🔄 Updated template to nf-core/tools v2.10
• 🔧 Updated nf-core modules for FastQC, samtools sort, samtools flagstat @ewels
  • ❌ Removes problematic -m memory assignment for samtools sort #81
• 🧾 Use fromSamplesheet from nf-validation #341 @Emiller88
• 🚀 Update Maintainers and add CODEOWNERS #345 @Emiller88
• ⚙️ Update schema to utilize exists and add more patterns #342 @Emiller88
• 📁 Support pipeline-specific configs #343 @maxulysse

Bug fixes & refactoring

• 🛠️ Added publishing of coverage (*cov.gz) files for NOMe-seq filtered reads for coverage2cytosine @FelixKrueger
• 🛠️ Wrong display values for “zymo” and “em_seq” presets on help page #335 @wassimsalam01
• 📚 Use new Citation tools functions #336 @Emiller88

Full Changelog: 2.4.0…2.5.0

Pipeline Updates

• Updated template to nf-core/tools v2.8
• Add --bamqc_regions_file parameter for targeted methylation sequencing data #302
• ✨ Add NF-TEST tests and snapshots for the pipeline test profile #310

Bug fixes & refactoring

• 🛠️ update index file channels to explicit value channels #310
• 🐛 fix params.test_data_base in test and test_full configs #310
• 🤖 GitHub Actions CI - pull_reqest to dev tests with NXF_VER latest-everything #310
• 🤖 GitHub Actions CI - pull_reqest to master tests with NXF_VER 22.10.1 & latest-everything #310
• 🤖 GitHub Actions CI - fail-fast set to false #310
• 🐛 get to the bottom of index tests #278
• ✨ Support for Bismark methylation extraction ignore and ignore_3prime parameters when ignore_r1 or ignore_3prime_r1 are greater than 0. #322
• 🛠️ rename ignore -> ignore_r1 and ignore_3prime -> ignore_3prime_r1 params #322
• 🐛 removed unused directory #297

Pipeline Updates

• ⚙️ Dramatically increase the default process time config requests for Bismark and bwa meth alignment
• ✨ Add a tower.yml file to enable Reports in Nextflow Tower
• 🤖 GitHub Actions CI - download the test data prior to running tests

Bug fixes & refactoring

• 🧹 Refactor genome indices preparation into a separate workflow
• 🧹 Refactor subworkflow logic out of alignment subworkflows, for later sharing
• 🐛 Fix a bug with using a local genome reference FASTA file
• 🐛 Fix a bunch of problems in the CI tests using nf-test (#279)

Pipeline Updates

• ✨ Updated the bismark2summary step so that it no longer stages the aligned BAM files into the working directory (#268)
  • Should be much faster / cheaper for running on the cloud
• ✨ Added ability to merge FastQ files based on shared IDs in sample sheet (#272)

Bug fixes & refactoring

• 🐛 Fixed typo in parameter handling for input reference indices (#263)
• 🧹 Removed orphaned --bismark_align_cpu_per_multicore and --bismark_align_cpu_per_multicore parameters.
  • Multi-core usage for Bismark alignment is now automatically set. If you would like to overwrite this, you can do so by setting ext.args for the process in a custom config.
• 🧹 Removed duplicate option --coverage2cytosine (#273)
  • Use the existing option --cytosine_report to launch the new COVERAGE2CYTOSINE process.
  • Removed option --cytosine_report genome_index from the Bismark methylation extractor.

Pipeline Updates

• ✨ Added option to run the Bismark coverage2cytosine script using the --coverage2cytosine and --nomeseq parameters.
• 🐛 Fixed bad bug where trimming presets were not being applied (#261)

Software Updates

• Update Bismark v0.23.0 to v0.24.0

Pipeline Updates

Major pipeline rewrite to use DSL2 with shared nf-core/modules.

Warning: Breaking change! ⚠️

The pipeline now requires a sample sheet to be passed to the pipeline with --input:

See an example here

Note: The genome column is not yet used but will give the ability to map to multiple genomes in a single run in a future release. See #181.

Supplying the reference geneome with --genome as before works as usual.

Software Updates

Major updates in commands and software versions for nearly every tool.

Please treat this new version with a little more care than usual and let us know if you find any problems!

Minor release to fix a bug in the command for the Bismark alignment. This change should have no effect on the pipeline output, though single-end alignments with Bismark should now be 2x faster. See issue #210 for full investigation.

Pipeline Updates

• Added new config profile to run minimal test paired-end dataset, with -profile test_paired. Added to the CI tests.

Bug fixes

• Fixed silent bug in Bismark alignment command that had no effect on the output (#210)

Software updates

• Picard 2.25.1 > 2.25.4
• MultiQC 1.10 > 1.10.1

⚠️ Breaking change!

In line with a standardisation change across all of nf-core, we have changed the main parameter name for supplying files to the pipeline. In this release, please use --input instead of --reads. The parameter still works in the same way as before.

Pipeline Updates

• Increased resources for fastqc process (#143)
• Raised Nextflow version requirement to 20.07.1
• Updated template to nf-core/tools 1.13.3
• Renamed --reads to --input
• Added new --maxins and --minins parameters to pass on to Bismark
• New --em_seq preset
  • Sets bismark_maxins = 1000, clip_r1 = 8, clip_r2 = 8, three_prime_clip_r1 = 8, three_prime_clip_r2 = 8
• New --publish_dir_mode parameter to customise results folder behaviour
• Fix bug on AWS for bismark_hisat known splice file (#177)
• Moved parameter documentation into new nextflow_schema.json file
  • This improves web documentation and enables nf-core launch functionality. See https://nf-co.re/launch?pipeline=methylseq
• Added a -profile test_full config for running the pipeline with a full-size test dataset
  • See the config file for details
  • This will be used for automated release tests on AWS, results browsable on the website

Software updates

• Python base 3.7.3 > 3.8.8
• markdown 3.1.1 > 3.3.4
• pymdown-extensions 6.0 > 8.1.1
• pygments 2.6.1 > 2.8.1
• pigz 2.3.4 > 2.6
• samtools 1.9 > 1.11
• TrimGalore! 0.6.5 > 0.6.6
• Bowtie2 2.3.5 > 2.4.2
• Hisat2 2.2.0 > 2.2.1
• Bismark 0.22.3 > 0.23.0
• Picard 2.22.2 > 2.25.1
• MethylDackel 0.5.0 > 0.5.2
• MultiQC 1.8 > 1.10

New features

• Added multicore support for TrimGalore!
• Improved the multicore support for Bismark methXtract for more parallelisation (#121)
• Added --cytosine_report option to tell Bismark to give reports for all cytosines in the genome.
• Added options --bismark_align_cpu_per_multicore and --bismark_align_cpu_per_multicore to customise how Bismark align --multicore is decided (#124)

Software updates

• New: pigz 2.3.4
• Python base 2.7 > 3.7.3
• FastQC 0.11.8 > 0.11.9
• TrimGalore! 0.6.4 > 0.6.5
• HiSAT2 2.1.0 > 2.2.0
• Bismark 0.22.2 > 0.22.3
• Qualimap 2.2.2c > 2.2.2d
• Picard 2.21.3 > 2.22.2
• MethylDackel 0.4.0 > 0.5.0
• MultiQC 1.7 > 1.8

Pipeline Updates

• Fixed bug where the iGenomes config was loaded after the references were set (#121)
• Removed awsbatch config profile because it is now served by nf-core/configs
• Tidied up the summary log messages when starting the pipeline
  • Fewer messages saying what you’re not doing, sanitised the order of some logs and removed a few things
• Slightly refactored the code for trimming parameters
• Updated template to tools 1.9

Bug fixes

• Fixed error where MethylDackel would consume the Nextflow channels and not work with more than one sample #140

New features

• Changed all parameter names to snake_case
• Added --local_alignment option to run Bismark with the --local flag to allow soft-clipping of reads.
• Added support for bismark’s SLAM-seq mode
• Added support for running bismark with HISAT2 as an aligner option #85
• Added support for centralized configuration profiles nf-core/configs
• Add --meth_cutoff parameter to change default for bismark_methylation_extractor
  • eg. use --meth_cutoff 5 on the command line or params.meth_cutoff = 5 to require 5 overlapping reads to call a methylation site.
• Added --methyl_kit option to run MethylDackel with the --methylKit flag, producing output suitable for the methylKit R package.

Software updates

• new dependency: hisat2 2.1.0
• new dependency: r-markdown 1.1
• TrimGalore! 0.5.0 > 0.6.4
• Bismark 0.20.0 > 0.22.2
• Bowtie2 2.3.4.3 > 2.3.5
• Picard 2.18.21 > 2.21.3
• Qualimap 2.2.2b > 2.2.2c
• MethylDackel 0.3.0 > 0.4.0

Pipeline updates

• Keep memory in GB for samtools, to avoid problems with unit conversion (#99)
• Changed params.container for process.container
• Synchronised with version 1.7 of the nf-core/tools template

Bug fixes

• Fixed a bug that caused conda dependencies to be resolved very slowly
• Allowed some spare memory in the samtools sort steps, avoiding crashes for some users (#81)

A fairly quick release on the heels of 1.2 to fix a nasty bug introduced that prevented the pipeline from running on more than one file. Whilst we’re here we added in a new tool: preseq.

New features

• Added preseq analysis to calculate sample complexity.
  • This new step can help decide sufficient sequencing depth has been reached.

Bug fixes

• Fixed new bug that meant pipeline only worked with one sample at a time #66
  • Introduced in previous release. TrimGalore onwards would only process one sample.

New features

• Trim 9bp from both ends of both reads for PBAT mode.
• Save where_are_my_files.txt to results directory to inform the user about missing intermediate files (#42)

Software updates

• Fastqc 0.11.7 > 0.11.8
• Bowtie2 2.3.4.2 > 2.3.4.3
• Bismark 0.19.1 > 0.20.0
• Qualimap 2.2.2a > 2.2.2b
• Picard 2.18.11 > 2.18.21
• MultiQC 1.6 > 1.7

Bug fixes

• Fixed error when running the pipeline with --unmapped
  • Previously, could result in error Error ~ No such variable: bismark_unmapped
• Fixed error where single-sample reports could mix up log files (#48)
• Fixed bug in MultiQC process that skipped results from some tools
• Supply available memory as argument to Picard MarkDuplicates

• Tests simplified - now work by simply using the test config profile
  • eg: nextflow run nf-core/methylseq -profile test,docker
  • Removed previous run_test.sh script and data
• New Singularity build script for direct compatibility with singularity-hub
• Minor improvements to the docs
• A number of boilerplate nf-core code updates
• Updated process$name nextflow syntax to avoid warnings in new versions of nextflow
• Updated software tools
  • trim-galore v0.4.5 update to 0.5.0
  • samtools v1.8 update to 1.9
  • bowtie2 v2.3.4.1 update to 2.3.4.2
  • multiqc v1.5 update to 1.6
  • picard v2.18.2 update to 2.18.11
  • bwameth v0.2.0 update to 0.2.2

First release of this pipeline under the nf-core flag. It also marks a significant step up in the maturity of the workflow, with everything now in a single script and both aligner workflows fully supported.

• Renamed and moved SciLifeLab/NGI-MethylSeq to nf-core/methylseq
• Merged bwa-meth and bismark pipeline scripts, now chosen with --aligner flag
• Refactored multi-core parameters for Bismark alignment and methylation extraction
• Rewrote most of the documentation
• Changed the Docker container to use Bioconda installations

For earlier releases and development notes, please see the (now archived) fork at SciLifeLab/NGI-MethylSeq.