Skip to content

Update expected files for all tests in yml files #649

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
FriederikeHanssen merged 42 commits into from
Jul 19, 2022
Merged

Update expected files for all tests in yml files #649

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
770881c
update aligner tests expected files
maxulysse Jul 17, 2022
ee4b556
Merge remote-tracking branch 'upstream/dev' into dev_tests_yml
maxulysse Jul 18, 2022
4ecd84e
fix output for targete bed
maxulysse Jul 18, 2022
c39047a
code polish
maxulysse Jul 18, 2022
889768d
fix expected outputs
maxulysse Jul 18, 2022
ad44d81
fix typo
maxulysse Jul 18, 2022
4f6d0cc
update annotation tests
maxulysse Jul 18, 2022
0b89443
update alignment_to_fastq tests
maxulysse Jul 18, 2022
7031b11
update default tests expected files
maxulysse Jul 18, 2022
4693f95
update gatk_spark tests
maxulysse Jul 18, 2022
2e512cd
no need for index there
maxulysse Jul 18, 2022
82b39f4
Merge remote-tracking branch 'upstream/dev' into dev_tests_yml
maxulysse Jul 18, 2022
7a07ca0
removing strelka from markduplicates tests
maxulysse Jul 18, 2022
d98205d
actually index is needed, but not always
maxulysse Jul 18, 2022
0c11aab
update markduplicates tests
maxulysse Jul 18, 2022
5c67a8f
update only paired VC tests
maxulysse Jul 18, 2022
d50e27b
update only paired variantcalling tests
maxulysse Jul 18, 2022
3e99d7f
catch WARN messages
maxulysse Jul 18, 2022
64ce0a6
fix fastqc output tests
maxulysse Jul 18, 2022
35f518c
improve test_pair and test_prepare_recalibration
maxulysse Jul 18, 2022
94ccbc0
fix typo
maxulysse Jul 18, 2022
49d3eca
update recalibrate tests
maxulysse Jul 18, 2022
afdcd40
update save output bam and bam mapped tests
maxulysse Jul 18, 2022
c488ed3
stash
maxulysse Jul 18, 2022
c46372b
Merge remote-tracking branch 'upstream/dev' into dev_tests_yml
maxulysse Jul 18, 2022
93b241b
forgot conflicts
maxulysse Jul 18, 2022
e9a3362
why 2 bams in single_bam?
maxulysse Jul 18, 2022
68f61ae
update CHANGELOG
maxulysse Jul 18, 2022
eec5eb0
channel magic for skip or not markduplicates
maxulysse Jul 18, 2022
20db917
fix tests
maxulysse Jul 18, 2022
e4a2ab1
improve csv creation when skipping markduplicates + update tests
maxulysse Jul 18, 2022
82e32d1
fix paths
maxulysse Jul 18, 2022
934a0dd
fix logic
maxulysse Jul 18, 2022
3653b9e
fix skip_qc tests
maxulysse Jul 19, 2022
e7a3aad
Merge remote-tracking branch 'upstream/dev' into dev_tests_yml
maxulysse Jul 19, 2022
98cc11e
update more tests
maxulysse Jul 19, 2022
b71b726
fix path after merging
maxulysse Jul 19, 2022
1400e2b
update all tools tests
maxulysse Jul 19, 2022
700b30d
Merge remote-tracking branch 'upstream/dev' into dev_tests_yml
maxulysse Jul 19, 2022
e5b6131
fix mpileup tests
maxulysse Jul 19, 2022
089dc55
Merge remote-tracking branch 'upstream/dev' into dev_tests_yml
maxulysse Jul 19, 2022
a1b3e0a
fix cnvkit
maxulysse Jul 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion conf/modules.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ process {
}

withName: 'TABIX_BGZIPTABIX' {
ext.prefix = {"${meta.id}.bed"}
ext.prefix = {"${meta.id}"}
publishDir = [
enabled: params.save_reference,
mode: params.publish_dir_mode,
Expand Down
1 change: 0 additions & 1 deletion conf/test.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ try {
}

params {

config_profile_name = 'Test profile'
config_profile_description = 'Minimal test dataset to check pipeline function'

Expand Down
32 changes: 26 additions & 6 deletions tests/test_aligner.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@
- bwa-mem2
- preprocessing
files:
- path: results/multiqc
- path: results/csv/markduplicates.csv
- path: results/csv/markduplicates_no_table.csv
- path: results/csv/recalibrated.csv
- path: results/multiqc
- path: results/preprocessing/test/markduplicates/test.md.cram
- path: results/preprocessing/test/markduplicates/test.md.cram.crai
- path: results/preprocessing/test/recal_table/test.recal.table
Expand All @@ -19,15 +19,25 @@
- path: results/reference/bwamem2/genome.fasta.ann
- path: results/reference/bwamem2/genome.fasta.bwt.2bit.64
- path: results/reference/bwamem2/genome.fasta.pac
- path: results/reference/dbsnp/dbsnp_146.hg38.vcf.gz.tbi
- path: results/reference/fai/genome.fasta.fai
- path: results/reference/gatk4/genome.dict
- path: results/reference/intervals/chr22_1-40001.bed
- path: results/reference/intervals/genome.bed
- path: results/reference/known_indels/mills_and_1000G.indels.vcf.gz.tbi
- path: results/reference/target/chr22_1-40001.bed.gz
- path: results/reports/fastqc/test-test_L1
- path: results/reports/markduplicates/test/test.md.metrics
- path: results/reports/mosdepth/test/test.md.mosdepth.global.dist.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.region.dist.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.md.regions.bed.gz
- path: results/reports/mosdepth/test/test.md.regions.bed.gz.csi
- path: results/reports/mosdepth/test/test.recal.mosdepth.global.dist.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.region.dist.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.recal.regions.bed.gz
- path: results/reports/mosdepth/test/test.recal.regions.bed.gz.csi
- path: results/reports/samtools_stats/test/test.md.cram.stats
- path: results/reports/samtools_stats/test/test.recal.cram.stats
- name: Run dragmap
Expand All @@ -37,15 +47,16 @@
- dragmap
- preprocessing
files:
- path: results/multiqc
- path: results/csv/markduplicates.csv
- path: results/csv/markduplicates_no_table.csv
- path: results/csv/recalibrated.csv
- path: results/multiqc
- path: results/preprocessing/test/markduplicates/test.md.cram
- path: results/preprocessing/test/markduplicates/test.md.cram.crai
- path: results/preprocessing/test/recal_table/test.recal.table
- path: results/preprocessing/test/recalibrated/test.recal.cram
- path: results/preprocessing/test/recalibrated/test.recal.cram.crai
- path: results/reference/dbsnp/dbsnp_146.hg38.vcf.gz.tbi
- path: results/reference/dragmap/hash_table.cfg
- path: results/reference/dragmap/hash_table.cfg.bin
- path: results/reference/dragmap/hash_table.cmp
Expand All @@ -54,14 +65,23 @@
- path: results/reference/dragmap/reference.bin
- path: results/reference/dragmap/repeat_mask.bin
- path: results/reference/dragmap/str_table.bin
- path: results/reference/fai/genome.fasta.fai
- path: results/reference/gatk4/genome.dict
- path: results/reference/intervals/chr22_1-40001.bed
- path: results/reference/intervals/genome.bed
- path: results/reference/known_indels/mills_and_1000G.indels.vcf.gz.tbi
- path: results/reference/target/chr22_1-40001.bed.gz
- path: results/reports/fastqc/test-test_L1
- path: results/reports/markduplicates/test/test.md.metrics
- path: results/reports/mosdepth/test/test.md.mosdepth.global.dist.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.region.dist.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.md.regions.bed.gz
- path: results/reports/mosdepth/test/test.md.regions.bed.gz.csi
- path: results/reports/mosdepth/test/test.recal.mosdepth.global.dist.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.region.dist.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.recal.regions.bed.gz
- path: results/reports/mosdepth/test/test.recal.regions.bed.gz.csi
- path: results/reports/samtools_stats/test/test.md.cram.stats
- path: results/reports/samtools_stats/test/test.recal.cram.stats
40 changes: 38 additions & 2 deletions tests/test_annotation.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@
files:
- path: results/annotation/test/test_snpEff.ann.vcf.gz
- path: results/annotation/test/test_snpEff.ann.vcf.gz.tbi
- path: results/reports/snpeff/test/test_snpEff.csv
- path: results/multiqc
- path: results/reports/snpeff/test/snpEff_summary.html
- path: results/reports/snpeff/test/test_snpEff.csv
- path: results/reports/snpeff/test/test_snpEff.genes.txt
- path: results/multiqc
- name: Run VEP
command: nextflow run main.nf -profile test,annotation,docker --tools vep --skip_tools multiqc
tags:
Expand All @@ -29,6 +29,42 @@
files:
- path: results/annotation/test/test_snpEff_VEP.ann.vcf.gz
- path: results/annotation/test/test_snpEff_VEP.ann.vcf.gz.tbi
- path: results/reports/EnsemblVEP/test/test_snpEff_VEP.summary.html
- path: results/annotation/test/test_snpEff.ann.vcf.gz
should_exist: false
- path: results/annotation/test/test_snpEff.ann.vcf.gz.tbi
should_exist: false
- path: results/annotation/test/test_VEP.ann.vcf.gz
should_exist: false
- path: results/annotation/test/test_VEP.ann.vcf.gz.tbi
should_exist: false
- path: results/reports/snpeff/test/snpEff_summary.html
should_exist: false
- path: results/reports/snpeff/test/test_snpEff.csv
should_exist: false
- path: results/reports/snpeff/test/test_snpEff.genes.txt
should_exist: false
- path: results/reports/EnsemblVEP/test/test_VEP.summary.html
should_exist: false
- name: Run snpEff, VEP and both consecutively
command: nextflow run main.nf -profile test,annotation,docker --tools merge,snpeff,vep --skip_tools multiqc
tags:
- annotation
- merge
- snpeff
- vep
files:
- path: results/annotation/test/test_VEP.ann.vcf.gz
- path: results/annotation/test/test_VEP.ann.vcf.gz.tbi
- path: results/annotation/test/test_snpEff.ann.vcf.gz
- path: results/annotation/test/test_snpEff.ann.vcf.gz.tbi
- path: results/annotation/test/test_snpEff_VEP.ann.vcf.gz
- path: results/annotation/test/test_snpEff_VEP.ann.vcf.gz.tbi
- path: results/reports/EnsemblVEP/test/test_VEP.summary.html
- path: results/reports/EnsemblVEP/test/test_snpEff_VEP.summary.html
- path: results/reports/snpeff/test/snpEff_summary.html
- path: results/reports/snpeff/test/test_snpEff.csv
- path: results/reports/snpeff/test/test_snpEff.genes.txt
- name: Run VEP with fasta
command: nextflow run main.nf -profile test,annotation,docker --tools vep --vep_include_fasta --skip_tools multiqc
tags:
Expand Down
40 changes: 33 additions & 7 deletions tests/test_bam_remap.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,42 @@
command: nextflow run main.nf -profile test,alignment_to_fastq,docker
tags:
- alignment_to_fastq

files:
- path: results/cat/test-1_1.merged.fastq.gz
- path: results/cat/test-1_2.merged.fastq.gz
- path: results/collate/test-1.mapped_1.fq.gz
- path: results/collate/test-1.mapped_2.fq.gz
- path: results/collate/test-1.mapped_other.fq.gz
- path: results/collate/test-1.mapped_singleton.fq.gz
- path: results/collate/test-1.unmapped_1.fq.gz
- path: results/collate/test-1.unmapped_2.fq.gz
- path: results/collate/test-1.unmapped_other.fq.gz
- path: results/collate/test-1.unmapped_singleton.fq.gz
- path: results/csv/markduplicates.csv
- path: results/csv/markduplicates_no_table.csv
- path: results/csv/recalibrated.csv
- path: results/multiqc/multiqc_report.html
- path: results/pipeline_info
- path: results/preprocessing/test
- path: results/reports
- path: results/samtools
- path: results/collate
- path: results/multiqc
- path: results/preprocessing/test/markduplicates/test.md.cram
- path: results/preprocessing/test/markduplicates/test.md.cram.crai
- path: results/preprocessing/test/recal_table/test.recal.table
- path: results/preprocessing/test/recalibrated/test.recal.cram
- path: results/preprocessing/test/recalibrated/test.recal.cram.crai
- path: results/reports/fastqc/test-1
- path: results/reports/markduplicates/test/test.md.metrics
- path: results/reports/mosdepth/test/test.md.mosdepth.global.dist.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.region.dist.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.md.regions.bed.gz
- path: results/reports/mosdepth/test/test.md.regions.bed.gz.csi
- path: results/reports/mosdepth/test/test.recal.mosdepth.global.dist.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.region.dist.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.recal.regions.bed.gz
- path: results/reports/mosdepth/test/test.recal.regions.bed.gz.csi
- path: results/reports/samtools_stats/test/test.md.cram.stats
- path: results/reports/samtools_stats/test/test.recal.cram.stats
- path: results/samtools/test-1.bam
- path: results/samtools/test-1.map_map.bam
- path: results/samtools/test-1.map_unmap.bam
- path: results/samtools/test-1.unmap_map.bam
- path: results/samtools/test-1.unmap_unmap.bam
54 changes: 34 additions & 20 deletions tests/test_default.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,84 +4,98 @@
- default
- preprocessing
files:
- path: results/csv/markduplicates.csv
- path: results/csv/markduplicates_no_table.csv
- path: results/csv/recalibrated.csv
- path: results/multiqc
- path: results/preprocessing/test/markduplicates/test.md.cram
- path: results/preprocessing/test/markduplicates/test.md.cram.crai
- path: results/preprocessing/test/recal_table/test.recal.table
- path: results/preprocessing/test/recalibrated/test.recal.cram
- path: results/preprocessing/test/recalibrated/test.recal.cram.crai
- path: results/csv/markduplicates.csv
- path: results/csv/markduplicates_no_table.csv
- path: results/csv/recalibrated.csv
- path: results/reports/fastqc/test-test_L1
- path: results/reports/fastqc
- path: results/reports/markduplicates/test/test.md.metrics
- path: results/reports/mosdepth/test/test.md.mosdepth.global.dist.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.region.dist.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.md.regions.bed.gz
- path: results/reports/mosdepth/test/test.md.regions.bed.gz.csi
- path: results/reports/mosdepth/test/test.recal.mosdepth.global.dist.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.region.dist.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.recal.regions.bed.gz
- path: results/reports/mosdepth/test/test.recal.regions.bed.gz.csi
- path: results/reports/samtools_stats/test/test.md.cram.stats
- path: results/reports/samtools_stats/test/test.recal.cram.stats

- name: Run default pipeline without intervals
command: nextflow run main.nf -profile test,no_intervals,docker
tags:
- default
- no_intervals
- preprocessing
files:
- path: results/csv/markduplicates.csv
- path: results/csv/markduplicates_no_table.csv
- path: results/csv/recalibrated.csv
- path: results/multiqc
- path: results/no_intervals.bed
- path: results/no_intervals.bed.gz
- path: results/no_intervals.bed.gz.tbi
- path: results/preprocessing/test/markduplicates/test.md.cram
- path: results/preprocessing/test/markduplicates/test.md.cram.crai
- path: results/preprocessing/test/recal_table/test.recal.table
- path: results/preprocessing/test/recalibrated/test.recal.cram
- path: results/preprocessing/test/recalibrated/test.recal.cram.crai
- path: results/csv/markduplicates.csv
- path: results/csv/markduplicates_no_table.csv
- path: results/csv/recalibrated.csv
- path: results/reports/fastqc/test-test_L1
- path: results/reports/fastqc
- path: results/reports/markduplicates/test/test.md.metrics
- path: results/reports/mosdepth/test/test.md.mosdepth.global.dist.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.region.dist.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.md.regions.bed.gz
- path: results/reports/mosdepth/test/test.md.regions.bed.gz.csi
- path: results/reports/mosdepth/test/test.recal.mosdepth.global.dist.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.region.dist.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.recal.regions.bed.gz
- path: results/reports/mosdepth/test/test.recal.regions.bed.gz.csi
- path: results/reports/samtools_stats/test/test.md.cram.stats
- path: results/reports/samtools_stats/test/test.recal.cram.stats

- name: Run default pipeline with VC
command: nextflow run main.nf -profile test,docker --tools strelka
tags:
- default
- preprocessing
- variant_calling
- strelka
files:
- path: results/csv/markduplicates.csv
- path: results/csv/markduplicates_no_table.csv
- path: results/csv/recalibrated.csv
- path: results/csv/variantcalled.csv
- path: results/multiqc
- path: results/preprocessing/test/markduplicates/test.md.cram
- path: results/preprocessing/test/markduplicates/test.md.cram.crai
- path: results/preprocessing/test/recal_table/test.recal.table
- path: results/preprocessing/test/recalibrated/test.recal.cram
- path: results/preprocessing/test/recalibrated/test.recal.cram.crai
- path: results/csv/markduplicates.csv
- path: results/csv/markduplicates_no_table.csv
- path: results/csv/recalibrated.csv
- path: results/reports/fastqc/test-test_L1
- path: results/reports/bcftools/test/strelka/test.strelka.variants.bcftools_stats.txt
- path: results/reports/fastqc
- path: results/reports/markduplicates/test/test.md.metrics
- path: results/reports/mosdepth/test/test.md.mosdepth.global.dist.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.region.dist.txt
- path: results/reports/mosdepth/test/test.md.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.md.regions.bed.gz
- path: results/reports/mosdepth/test/test.md.regions.bed.gz.csi
- path: results/reports/mosdepth/test/test.recal.mosdepth.global.dist.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.region.dist.txt
- path: results/reports/mosdepth/test/test.recal.mosdepth.summary.txt
- path: results/reports/mosdepth/test/test.recal.regions.bed.gz
- path: results/reports/mosdepth/test/test.recal.regions.bed.gz.csi
- path: results/reports/samtools_stats/test/test.md.cram.stats
- path: results/reports/samtools_stats/test/test.recal.cram.stats
- path: results/reports/vcftools/test/strelka/test.strelka.variants.FILTER.summary
- path: results/reports/vcftools/test/strelka/test.strelka.variants.TsTv.count
- path: results/reports/vcftools/test/strelka/test.strelka.variants.TsTv.qual
- path: results/variant_calling/test/strelka/test.strelka.genome.vcf.gz
- path: results/variant_calling/test/strelka/test.strelka.genome.vcf.gz.tbi
- path: results/variant_calling/test/strelka/test.strelka.variants.vcf.gz
Expand Down