Porting pipeline to DSL2 by SusiJo · Pull Request #43 · nf-core/bamtofastq

SusiJo · 2022-10-21T10:56:59Z

Many thanks to contributing to qbic-pipelines/bamtofastq!

Please fill in the appropriate checklist below (delete whatever is not relevant). These are the most common things requested on pull requests (PRs).

PR checklist

This comment contains a description of changes (with reason)
If you've fixed a bug or added code that should be tested, add tests!
If necessary, add test data to testdata/
Ensure the test suite passes (nextflow run . -profile test,docker).
Make sure your code lints (nf-core lint .).
Documentation in docs is updated
CHANGELOG.md is updated
README.md is updated

Learn more about contributing: https://github.com/qbic-pipelines/bamtofastq/tree/master/.github/CONTRIBUTING.md

SusiJo · 2022-10-21T11:02:27Z

Hi @FriederikeHanssen,
This is still a Draft PR so no need for a review yet. Since I had some branch issues on the other PR, I decided to make a fresh and cleaner PR. Also we decided to try integrating the pipeline into nf-core, therefore the labels have changed 😉

Co-authored-by: FriederikeHanssen <FriederikeHanssen@users.noreply.github.com>

Co-authored-by: FriederikeHanssen <FriederikeHanssen@users.noreply.github.com> Co-authored-by: Gisela Gabernet <gisela.gabernet@qbic.uni-tuebingen.de>

Co-authored-by: FriederikeHanssen <FriederikeHanssen@users.noreply.github.com>

FriederikeHanssen · 2022-12-15T09:26:28Z

it would probably be good if we start merging PRs soon-ish to the dsl2, so we can fix things independently. Otherwise, I will have to push to your branch which will cause chaos :D

SusiJo · 2023-03-20T09:28:38Z

Linting is passing locally with nf-core tools 2.8.dev0. Might be failing because of a missing == in the pipeline template in the master branch that was added on the dev already. See here.

matrulda

Great job! I've left a couple of comments.

matrulda · 2023-03-28T07:49:58Z

.github/PULL_REQUEST_TEMPLATE.md

+
+- [ ] This comment contains a description of changes (with reason).
+- [ ] If you've fixed a bug or added code that should be tested, add tests!
+- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/bamtofastq/tree/master/.github/CONTRIBUTING.md)- [ ] If necessary, also make a PR on the nf-core/bamtofastq _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.


Suggested change

- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/bamtofastq/tree/master/.github/CONTRIBUTING.md)- [ ] If necessary, also make a PR on the nf-core/bamtofastq _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.

- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/bamtofastq/tree/master/.github/CONTRIBUTING.md)

- [ ] If necessary, also make a PR on the nf-core/bamtofastq _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.

That's actually a typo in the pipeline template :D

matrulda · 2023-03-28T12:19:45Z

assets/multiqc_config.yaml

Should this file be removed? It looks like multiqc_config.yml is the one that is used. In that case, should perhaps the top modules info should be moved to that config?

matrulda · 2023-03-29T08:13:14Z

docs/usage.md

+| `sample`   | Custom sample name.                                                                                                     |
+| `mapped`   | Full path to input BAM/CRAM file. File has to have the extension ".bam" or ".cram".                                     |
+| `index`    | If available provide full path to input BAI/CRAI index file. File has to have the extension ".bam.bai" or ".cram.crai". |
+| `filetype` | For input BAM files the filetype hast to be "bam" and for input CRAM files, the filetype needs to be "cram".            |


Perhaps the description can simply be something like: "Type of input file. Options: bam or cram"

matrulda · 2023-03-29T08:24:36Z

docs/usage.md

-Unfortunately, at the time of writing, FastQC [doesn't support](https://github.com/s-andrews/FastQC/issues/54) CRAM files as input. Hence, a benefit of converting CRAM files to BAM format as opposed to converting directly to FASTQ format is that you can perform QC before the final conversion.
+### `--fasta`
+
+Use this option to indicate which reference genome FASTA file to use when decompressing CRAM files. It will only work if the reference genome FASTA file listed in the CRAM header is available (_e.g._ via HTTP/FTP or on the local file system). Otherwise, you will need to use the [`--fasta`](#--fasta) option. You can check which reference FASTA file is indicated in the CRAM header with the following command:


I think this section is a bit confusing. If I understand it right, this parameter is only needed if an accessible fasta file isn't specified the header?

Suggestion:
When converting a CRAM file the fasta file specified in the CRAM header is used to decompress the file. If that file isn't available, you will need to specify an alternative path using the --fasta option.

Thanks! Sounds much better!

matrulda · 2023-03-29T09:50:23Z

main.nf

+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+*/

+params.chr = WorkflowMain.getGenomeAttribute(params, 'chr')


Is this needed? Afaik this function is used to extract info from a genomes config (like igenomes.config). Don't feel like it's applicable here? I could be missing something though :)

Good spot, thanks!

matrulda · 2023-03-31T13:20:49Z

workflows/bamtofastq.nf

+include { SAMTOOLS_VIEW as SAMTOOLS_CHR        } from '../modules/nf-core/samtools/view/main'
+include { SAMTOOLS_VIEW as SAMTOOLS_PE         } from '../modules/nf-core/samtools/view/main'
+include { SAMTOOLS_INDEX as SAMTOOLS_CHR_INDEX } from '../modules/nf-core/samtools/index/main'
+include { CHECK_IF_PAIRED_END                  } from '../modules/local/check_paired_end'


Should this be imported in the above section?

workflows/bamtofastq.nf

matrulda · 2023-03-31T13:29:16Z

workflows/bamtofastq.nf

+                                                SAMTOOLS_CHR.out.cram)
+        SAMTOOLS_CHR_INDEX(samtools_chr_out)
+        ch_input = samtools_chr_out.join(Channel.empty().mix(SAMTOOLS_CHR_INDEX.out.bai,
+                                                            SAMTOOLS_CHR_INDEX.out.crai))


Align parameters?

workflows/bamtofastq.nf

matrulda · 2023-03-31T14:14:17Z

subworkflows/local/alignment_to_fastq.nf

+    // Index File if not PROVIDED -> this also requires updates to samtools view possibly URGH
+
+    // MAP - MAP
+    SAMTOOLS_VIEW_MAP_MAP(input, fasta, [])


I'm pretty sure I'm just missing something obvious, but I don't get how the samtools flags are passed to the module. I guess they are part of the meta data somehow, but I can't seem to find where.

They are specified in the modules.config for each named process.

SusiJo added 3 commits October 19, 2022 15:27

Initial template commit

1a2dd88

Porting pipeline to DSL2

0aae09f

prettier

db8d896

FriederikeHanssen mentioned this pull request Nov 2, 2022

Porting to DSL2 #42

Closed

8 tasks

SusiJo and others added 11 commits November 2, 2022 16:08

Template update for nf-core/tools version 2.6

58ed55e

merged new TEMPLATE

be8377f

add modules + subworkflow

08982e1

update modules

aeb9d26

Co-authored-by: FriederikeHanssen <FriederikeHanssen@users.noreply.github.com>

add samplesheet

b290cb8

Co-authored-by: FriederikeHanssen <FriederikeHanssen@users.noreply.github.com> Co-authored-by: Gisela Gabernet <gisela.gabernet@qbic.uni-tuebingen.de>

add samplesheet to test.config

7b11757

add sys.exit

57240d8

alignment workflow working :)

ec3add5

Co-authored-by: FriederikeHanssen <FriederikeHanssen@users.noreply.github.com>

add to aln workflow

a5e71a5

adjust schema

cf3dee9

ch gitignore

3af5d0f

SusiJo added 13 commits December 15, 2022 11:07

output fastq reads

a8b3006

added pre-qc

8dc7947

extract reads to chr working

bfd309d

fix naming chromosomes

3e12780

added post conversion QC

d05e5af

add multiqc report

55cb5d3

add single/paired detection + extraction

feac316

apply editorconfig + renaming

86ace66

apply editorconfig again

26f920f

update linting + prettierignore

5d5365b

test editorconfig

be28f55

rm jinja template string

bbe5f6d

editorconfig

56a84a1

SusiJo added 17 commits March 10, 2023 17:14

indent

0a1a9ea

change indent

7fc437d

add crams

4def56f

add prepare indices

0411aa8

add configs

4e2378b

compute fasta.fai if not provided

4b8c3af

update docs

e23b1d1

enabled flags

fff658e

updated modules

94a9878

add samplesheets [skip actions]

d5bbbde

Template update for nf-core/tools version 2.7.2

9a8c9b1

Template update for nf-core/tools version 2.8.dev0

f2c5daf

Merge branch 'TEMPLATE' into dls2_new

ab6ff21

fix updating template

44d212e

fix linting.yml

1a00915

prettier + changed paths

19e2085

new conda syntax + rm param

9fe7852

SusiJo marked this pull request as ready for review March 20, 2023 09:29

SusiJo merged commit 73c5542 into nf-core:dsl2 Mar 29, 2023

matrulda reviewed Mar 31, 2023

View reviewed changes

	- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/bamtofastq/tree/master/.github/CONTRIBUTING.md)- [ ] If necessary, also make a PR on the nf-core/bamtofastq _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
	- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/bamtofastq/tree/master/.github/CONTRIBUTING.md)
	- [ ] If necessary, also make a PR on the nf-core/bamtofastq _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.

Conversation

SusiJo commented Oct 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR checklist

Uh oh!

SusiJo commented Oct 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FriederikeHanssen commented Dec 15, 2022

Uh oh!

SusiJo commented Mar 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matrulda left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SusiJo Apr 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

SusiJo commented Oct 21, 2022 •

edited

Loading

SusiJo commented Oct 21, 2022 •

edited

Loading

SusiJo commented Mar 20, 2023 •

edited

Loading

SusiJo Apr 5, 2023 •

edited

Loading