Skip to content

Instantly share code, notes, and snippets.

@briandoconnor
Created July 16, 2017 08:03
Show Gist options
  • Select an option

  • Save briandoconnor/b4917355531533047d22bf0ba6076ea8 to your computer and use it in GitHub Desktop.

Select an option

Save briandoconnor/b4917355531533047d22bf0ba6076ea8 to your computer and use it in GitHub Desktop.
/home/ubuntu/envs/env-ga4gh-dream/bin/cwltool 1.0.20170217172322
Resolved 'main-NA12878-platinum-chr20.cwl' to 'file:///home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/main-NA12878-platinum-chr20.cwl'
[job prep_samples_to_rec] Output of job will be cached in /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/a5cb782c5a69dcd3836c6f6aaad47e4e
['docker', 'pull', 'quay.io/bcbio/bcbio-base']
Using default tag: latest
latest: Pulling from bcbio/bcbio-base
75c416ea735c: Pulling fs layer
c6ff40b6d658: Pulling fs layer
a7050fc1f338: Pulling fs layer
f0ffb5cf6ba9: Pulling fs layer
be232718519c: Pulling fs layer
2834eb77be41: Pulling fs layer
be232718519c: Waiting
2834eb77be41: Waiting
f0ffb5cf6ba9: Waiting
a7050fc1f338: Verifying Checksum
a7050fc1f338: Download complete
c6ff40b6d658: Verifying Checksum
c6ff40b6d658: Download complete
be232718519c: Download complete
f0ffb5cf6ba9: Verifying Checksum
f0ffb5cf6ba9: Download complete
75c416ea735c: Verifying Checksum
75c416ea735c: Download complete
75c416ea735c: Pull complete
c6ff40b6d658: Pull complete
a7050fc1f338: Pull complete
f0ffb5cf6ba9: Pull complete
be232718519c: Pull complete
2834eb77be41: Verifying Checksum
2834eb77be41: Download complete
2834eb77be41: Pull complete
Digest: sha256:c6a3a7a48fbc08805622b6a0fb9aae81b0b95408bc622b132dc75ea1bf200b23
Status: Downloaded newer image for quay.io/bcbio/bcbio-base:latest
[job prep_samples_to_rec] /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/a5cb782c5a69dcd3836c6f6aaad47e4e$ docker \
run \
-i \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa.fai:/var/lib/cwl/stg079eb279-7e1c-4478-b725-112b3fb39dda/hg38.fa.fai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa:/var/lib/cwl/stg079eb279-7e1c-4478-b725-112b3fb39dda/hg38.fa:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.dict:/var/lib/cwl/stg079eb279-7e1c-4478-b725-112b3fb39dda/hg38.dict:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/a5cb782c5a69dcd3836c6f6aaad47e4e:/var/spool/cwl:rw \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/tmp/UxegVa:/tmp:rw \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
--env=MPLCONFIGDIR=. \
quay.io/bcbio/bcbio-base \
bcbio_nextgen.py \
runfn \
prep_samples_to_rec \
cwl \
sentinel_runtime=cores,1,ram,3072 \
sentinel_parallel=multi-combined \
'sentinel_outputs=prep_samples_rec:description;reference__fasta__base;config__algorithm__variant_regions' \
sentinel_inputs=config__algorithm__variant_regions:var,reference__fasta__base:var,description:var
[job prep_samples_to_rec] completed success
[step prep_samples_to_rec] completed success
[job alignment_to_rec] Output of job will be cached in /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/fff9469d1a8ae685c1e99c2596513fb1
[job alignment_to_rec] /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/fff9469d1a8ae685c1e99c2596513fb1$ docker \
run \
-i \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/bwa-wf.tar.gz:/var/lib/cwl/stg524298a9-3851-4eca-986c-032fb8f9ad04/bwa-wf.tar.gz:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa:/var/lib/cwl/stgfb446077-83bf-42ec-ab3b-5ea0c6fc149f/hg38.fa:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/giab/na12878/NA12878-platinum-chr20.bam:/var/lib/cwl/stgf9a1d255-1613-495a-b06f-57abec18c4ba/NA12878-platinum-chr20.bam:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa.fai:/var/lib/cwl/stgfb446077-83bf-42ec-ab3b-5ea0c6fc149f/hg38.fa.fai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/giab/na12878/NA12878-platinum-chr20.bam.bai:/var/lib/cwl/stgf9a1d255-1613-495a-b06f-57abec18c4ba/NA12878-platinum-chr20.bam.bai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.dict:/var/lib/cwl/stgfb446077-83bf-42ec-ab3b-5ea0c6fc149f/hg38.dict:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/fff9469d1a8ae685c1e99c2596513fb1:/var/spool/cwl:rw \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/tmp/MChUAh:/tmp:rw \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
--env=MPLCONFIGDIR=. \
quay.io/bcbio/bcbio-base \
bcbio_nextgen.py \
runfn \
alignment_to_rec \
cwl \
sentinel_runtime=cores,1,ram,3072 \
sentinel_parallel=multi-combined \
'sentinel_outputs=alignment_rec:description;config__algorithm__align_split_size;reference__fasta__base;rgnames__lb;rgnames__rg;rgnames__lane;reference__bwa__indexes;files;config__algorithm__aligner;rgnames__pl;config__algorithm__mark_duplicates;rgnames__pu;rgnames__sample' \
sentinel_inputs=files:var,config__algorithm__align_split_size:var,reference__fasta__base:var,rgnames__pl:var,rgnames__sample:var,rgnames__pu:var,rgnames__lane:var,rgnames__rg:var,rgnames__lb:var,reference__bwa__indexes:var,config__algorithm__aligner:var,config__algorithm__mark_duplicates:var,description:var
[job alignment_to_rec] completed success
[step alignment_to_rec] completed success
[job prep_samples] Output of job will be cached in /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/e46ca5cb310cae6ef76a206ecfadb759
['docker', 'pull', 'quay.io/bcbio/bcbio-align']
Using default tag: latest
latest: Pulling from bcbio/bcbio-align
75c416ea735c: Already exists
c6ff40b6d658: Already exists
a7050fc1f338: Already exists
f0ffb5cf6ba9: Already exists
be232718519c: Already exists
2834eb77be41: Already exists
b8ab66395517: Pulling fs layer
74bad9141b53: Pulling fs layer
b8ab66395517: Download complete
b8ab66395517: Pull complete
74bad9141b53: Download complete
74bad9141b53: Pull complete
Digest: sha256:a5820aed2a672840cf6f5064b08e2da16380b2e1b1611ff7fa2951a46cf7805b
Status: Downloaded newer image for quay.io/bcbio/bcbio-align:latest
[job prep_samples] /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/e46ca5cb310cae6ef76a206ecfadb759$ docker \
run \
-i \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.dict:/var/lib/cwl/stgdfbc12b8-b8e0-4f9f-8637-7f5e7018297a/hg38.dict:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa:/var/lib/cwl/stgdfbc12b8-b8e0-4f9f-8637-7f5e7018297a/hg38.fa:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa.fai:/var/lib/cwl/stgdfbc12b8-b8e0-4f9f-8637-7f5e7018297a/hg38.fa.fai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/e46ca5cb310cae6ef76a206ecfadb759:/var/spool/cwl:rw \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/tmp/qeF5cf:/tmp:rw \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
--env=MPLCONFIGDIR=. \
quay.io/bcbio/bcbio-align \
bcbio_nextgen.py \
runfn \
prep_samples \
cwl \
sentinel_runtime=cores,1,ram,3072 \
sentinel_parallel=multi-parallel \
sentinel_outputs=config__algorithm__variant_regions,config__algorithm__variant_regions_merged,config__algorithm__variant_regions_orig,config__algorithm__coverage,config__algorithm__coverage_merged,config__algorithm__coverage_orig,config__algorithm__seq2c_bed_ready \
sentinel_inputs=prep_samples_rec:record
[job prep_samples] completed success
[step prep_samples] completed success
[job prep_align_inputs] Output of job will be cached in /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c49bfc76e3b35586f1622e2cacb0abc4
[job prep_align_inputs] /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c49bfc76e3b35586f1622e2cacb0abc4$ docker \
run \
-i \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/giab/na12878/NA12878-platinum-chr20.bam.bai:/var/lib/cwl/stg8dbbc15e-3084-4692-8667-f4ffe70ea0c3/NA12878-platinum-chr20.bam.bai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa:/var/lib/cwl/stg2859a5d9-6783-45c4-bf28-aded8646393f/hg38.fa:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/giab/na12878/NA12878-platinum-chr20.bam:/var/lib/cwl/stg8dbbc15e-3084-4692-8667-f4ffe70ea0c3/NA12878-platinum-chr20.bam:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.dict:/var/lib/cwl/stg2859a5d9-6783-45c4-bf28-aded8646393f/hg38.dict:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/bwa-wf.tar.gz:/var/lib/cwl/stg8969adcf-966c-4560-b16e-02a4ae5bfed7/bwa-wf.tar.gz:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa.fai:/var/lib/cwl/stg2859a5d9-6783-45c4-bf28-aded8646393f/hg38.fa.fai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c49bfc76e3b35586f1622e2cacb0abc4:/var/spool/cwl:rw \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/tmp/ZHttqy:/tmp:rw \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
--env=MPLCONFIGDIR=. \
quay.io/bcbio/bcbio-align \
bcbio_nextgen.py \
runfn \
prep_align_inputs \
cwl \
sentinel_runtime=cores,1,ram,3072 \
sentinel_parallel=single-split \
'sentinel_outputs=process_alignment_rec:files;config__algorithm__quality_format;align_split' \
sentinel_inputs=alignment_rec:record
[job prep_align_inputs] completed success
[step prep_align_inputs] completed success
[job process_alignment] Output of job will be cached in /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c8e5d3eb6ab662d94e97a019bcf73297
[job process_alignment] /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c8e5d3eb6ab662d94e97a019bcf73297$ docker \
run \
-i \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/giab/na12878/NA12878-platinum-chr20.bam.bai:/var/lib/cwl/stg86d1f146-f186-4144-8b68-01b2b8db76c5/NA12878-platinum-chr20.bam.bai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c49bfc76e3b35586f1622e2cacb0abc4/align_prep/NA12878-platinum-chr20-1.fq.gz.gbi:/var/lib/cwl/stg10ed02d6-2ef9-4244-bd1c-be5b9ea9088d/NA12878-platinum-chr20-1.fq.gz.gbi:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa:/var/lib/cwl/stge7fcc690-e3de-47b6-b7f2-bb59328098ef/hg38.fa:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/giab/na12878/NA12878-platinum-chr20.bam:/var/lib/cwl/stg86d1f146-f186-4144-8b68-01b2b8db76c5/NA12878-platinum-chr20.bam:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.dict:/var/lib/cwl/stge7fcc690-e3de-47b6-b7f2-bb59328098ef/hg38.dict:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/bwa-wf.tar.gz:/var/lib/cwl/stg8670b6c6-1130-4d17-8cd4-da4f0ff6fd1e/bwa-wf.tar.gz:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa.fai:/var/lib/cwl/stge7fcc690-e3de-47b6-b7f2-bb59328098ef/hg38.fa.fai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c49bfc76e3b35586f1622e2cacb0abc4/align_prep/NA12878-platinum-chr20-2.fq.gz.gbi:/var/lib/cwl/stg8f23aa18-9d8a-4772-a26e-7e818c5b8d23/NA12878-platinum-chr20-2.fq.gz.gbi:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c49bfc76e3b35586f1622e2cacb0abc4/align_prep/NA12878-platinum-chr20-2.fq.gz:/var/lib/cwl/stg8f23aa18-9d8a-4772-a26e-7e818c5b8d23/NA12878-platinum-chr20-2.fq.gz:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c49bfc76e3b35586f1622e2cacb0abc4/align_prep/NA12878-platinum-chr20-1.fq.gz:/var/lib/cwl/stg10ed02d6-2ef9-4244-bd1c-be5b9ea9088d/NA12878-platinum-chr20-1.fq.gz:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c8e5d3eb6ab662d94e97a019bcf73297:/var/spool/cwl:rw \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/tmp/cRJcb8:/tmp:rw \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
--env=MPLCONFIGDIR=. \
quay.io/bcbio/bcbio-align \
bcbio_nextgen.py \
runfn \
process_alignment \
cwl \
sentinel_runtime=cores,8,ram,24576 \
sentinel_parallel=single-parallel \
sentinel_outputs=work_bam,align_bam,hla__fastq,work_bam_plus__disc,work_bam_plus__sr \
sentinel_inputs=alignment_rec:record,process_alignment_rec:record
[job process_alignment] completed success
[step process_alignment] completed success
[job merge_split_alignments] Output of job will be cached in /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/5b7405849c774e668e972adec7efea63
[job merge_split_alignments] /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/5b7405849c774e668e972adec7efea63$ docker \
run \
-i \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c8e5d3eb6ab662d94e97a019bcf73297/align/NA12878-chr20/hla/NA12878-chr20-sort.bam.hla.HLA-B.fq:/var/lib/cwl/stg607db76a-8395-468c-924f-2f65ee027a25/NA12878-chr20-sort.bam.hla.HLA-B.fq:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c8e5d3eb6ab662d94e97a019bcf73297/align/NA12878-chr20/hla/NA12878-chr20-sort.bam.hla.HLA-DQB1.fq:/var/lib/cwl/stgce3a7ded-7bd3-4a70-b43c-f55b060227bb/NA12878-chr20-sort.bam.hla.HLA-DQB1.fq:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c8e5d3eb6ab662d94e97a019bcf73297/align/NA12878-chr20/hla/NA12878-chr20-sort.bam.hla.HLA-DRB1.fq:/var/lib/cwl/stg836b268b-04dd-44d2-b0a7-d444e20c9cbe/NA12878-chr20-sort.bam.hla.HLA-DRB1.fq:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/giab/na12878/NA12878-platinum-chr20.bam.bai:/var/lib/cwl/stg004a788c-a470-4d2a-b1ad-fe49b7f72d6e/NA12878-platinum-chr20.bam.bai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa:/var/lib/cwl/stg9bcdab40-24b3-4927-9dec-3e8c54fe6083/hg38.fa:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/giab/na12878/NA12878-platinum-chr20.bam:/var/lib/cwl/stg004a788c-a470-4d2a-b1ad-fe49b7f72d6e/NA12878-platinum-chr20.bam:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.dict:/var/lib/cwl/stg9bcdab40-24b3-4927-9dec-3e8c54fe6083/hg38.dict:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c8e5d3eb6ab662d94e97a019bcf73297/align/NA12878-chr20/NA12878-chr20-sort.bam:/var/lib/cwl/stgb628d0f2-606f-4ad1-8f9d-e4f463a36ed1/NA12878-chr20-sort.bam:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c8e5d3eb6ab662d94e97a019bcf73297/align/NA12878-chr20/hla/NA12878-chr20-sort.bam.hla.HLA-C.fq:/var/lib/cwl/stg35eeb882-28ce-4052-b949-123aef3b458f/NA12878-chr20-sort.bam.hla.HLA-C.fq:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/bwa-wf.tar.gz:/var/lib/cwl/stgd46daa30-4208-414e-9970-67a137aa0598/bwa-wf.tar.gz:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c8e5d3eb6ab662d94e97a019bcf73297/align/NA12878-chr20/NA12878-chr20-sort.bam.bai:/var/lib/cwl/stgb628d0f2-606f-4ad1-8f9d-e4f463a36ed1/NA12878-chr20-sort.bam.bai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa.fai:/var/lib/cwl/stg9bcdab40-24b3-4927-9dec-3e8c54fe6083/hg38.fa.fai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c8e5d3eb6ab662d94e97a019bcf73297/align/NA12878-chr20/hla/NA12878-chr20-sort.bam.hla.HLA-DQA1.fq:/var/lib/cwl/stg52414890-c8da-4417-9038-c67d623079fc/NA12878-chr20-sort.bam.hla.HLA-DQA1.fq:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/c8e5d3eb6ab662d94e97a019bcf73297/align/NA12878-chr20/hla/NA12878-chr20-sort.bam.hla.HLA-A.fq:/var/lib/cwl/stg99b695fc-c96d-45bc-832d-6e306364862e/NA12878-chr20-sort.bam.hla.HLA-A.fq:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/5b7405849c774e668e972adec7efea63:/var/spool/cwl:rw \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/tmp/qPQdac:/tmp:rw \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
--env=MPLCONFIGDIR=. \
quay.io/bcbio/bcbio-align \
bcbio_nextgen.py \
runfn \
merge_split_alignments \
cwl \
sentinel_runtime=cores,8,ram,24576 \
sentinel_parallel=single-merge \
sentinel_outputs=align_bam,work_bam_plus__disc,work_bam_plus__sr,hla__fastq \
sentinel_inputs=alignment_rec:record,work_bam:var,align_bam:var,work_bam_plus__disc:var,work_bam_plus__sr:var,hla__fastq:var
[job merge_split_alignments] completed success
[step merge_split_alignments] completed success
[workflow alignment] outdir is /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/D56GRI
[step alignment] completed success
[job postprocess_alignment_to_rec] Output of job will be cached in /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/893826a70af45bc599492435769d06ae
[job postprocess_alignment_to_rec] /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/893826a70af45bc599492435769d06ae$ docker \
run \
-i \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/5b7405849c774e668e972adec7efea63/align/NA12878-chr20/NA12878-chr20-sort.bam:/var/lib/cwl/stgd4712193-2336-4c73-beb8-e2161b6b9b1a/NA12878-chr20-sort.bam:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa.fai:/var/lib/cwl/stg249f0c8e-147f-41d9-b19d-0adef91bf990/hg38.fa.fai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa:/var/lib/cwl/stg249f0c8e-147f-41d9-b19d-0adef91bf990/hg38.fa:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.dict:/var/lib/cwl/stg249f0c8e-147f-41d9-b19d-0adef91bf990/hg38.dict:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/5b7405849c774e668e972adec7efea63/align/NA12878-chr20/NA12878-chr20-sort.bam.bai:/var/lib/cwl/stgd4712193-2336-4c73-beb8-e2161b6b9b1a/NA12878-chr20-sort.bam.bai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/893826a70af45bc599492435769d06ae:/var/spool/cwl:rw \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/tmp/EiUtIs:/tmp:rw \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
--env=MPLCONFIGDIR=. \
quay.io/bcbio/bcbio-base \
bcbio_nextgen.py \
runfn \
postprocess_alignment_to_rec \
cwl \
sentinel_runtime=cores,1,ram,3072 \
sentinel_parallel=multi-combined \
'sentinel_outputs=postprocess_alignment_rec:description;reference__fasta__base;config__algorithm__coverage_interval;config__algorithm__recalibrate;config__algorithm__variant_regions;align_bam;config__algorithm__variant_regions_merged;config__algorithm__variant_regions_orig;config__algorithm__coverage;config__algorithm__coverage_merged;config__algorithm__coverage_orig;config__algorithm__seq2c_bed_ready' \
sentinel_inputs=align_bam:var,config__algorithm__coverage_interval:var,config__algorithm__variant_regions:var,config__algorithm__variant_regions_merged:var,config__algorithm__variant_regions_orig:var,config__algorithm__coverage:var,config__algorithm__coverage_merged:var,config__algorithm__coverage_orig:var,config__algorithm__seq2c_bed_ready:var,config__algorithm__recalibrate:var,reference__fasta__base:var,description:var
[job postprocess_alignment_to_rec] completed success
[step postprocess_alignment_to_rec] completed success
[job postprocess_alignment] Output of job will be cached in /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/fc82baff4ab0229ad7895229e3fdb04f
[job postprocess_alignment] /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/fc82baff4ab0229ad7895229e3fdb04f$ docker \
run \
-i \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.dict:/var/lib/cwl/stg4719098a-36e6-4d0b-b5a3-7868d3dec072/hg38.dict:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa:/var/lib/cwl/stg4719098a-36e6-4d0b-b5a3-7868d3dec072/hg38.fa:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa.fai:/var/lib/cwl/stg4719098a-36e6-4d0b-b5a3-7868d3dec072/hg38.fa.fai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/5b7405849c774e668e972adec7efea63/align/NA12878-chr20/NA12878-chr20-sort.bam.bai:/var/lib/cwl/stgadf05344-26f4-4671-aa6a-7ab730b84596/NA12878-chr20-sort.bam.bai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/5b7405849c774e668e972adec7efea63/align/NA12878-chr20/NA12878-chr20-sort.bam:/var/lib/cwl/stgadf05344-26f4-4671-aa6a-7ab730b84596/NA12878-chr20-sort.bam:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/fc82baff4ab0229ad7895229e3fdb04f:/var/spool/cwl:rw \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/tmp/7KaFNz:/tmp:rw \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
--env=MPLCONFIGDIR=. \
quay.io/bcbio/bcbio-align \
bcbio_nextgen.py \
runfn \
postprocess_alignment \
cwl \
sentinel_runtime=cores,8,ram,24576 \
sentinel_parallel=multi-parallel \
sentinel_outputs=config__algorithm__coverage_interval,config__algorithm__variant_regions,config__algorithm__variant_regions_merged,config__algorithm__variant_regions_orig,config__algorithm__coverage,config__algorithm__coverage_merged,config__algorithm__coverage_orig,config__algorithm__seq2c_bed_ready,regions__callable,regions__sample_callable,regions__nblock,regions__highdepth \
sentinel_inputs=postprocess_alignment_rec:record
***** WARNING: File /var/spool/cwl/align/NA12878-chr20/NA12878-chr20-coverage.callable.bed has inconsistent naming convention for record:
HLA-A*01:01:01:01 0 3503 NO_COVERAGE
***** WARNING: File /var/spool/cwl/align/NA12878-chr20/NA12878-chr20-coverage.callable.bed has inconsistent naming convention for record:
HLA-A*01:01:01:01 0 3503 NO_COVERAGE
[job postprocess_alignment] completed success
[step postprocess_alignment] completed success
[job qc_to_rec] Output of job will be cached in /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/8803605effa5d60a41ef8b980ea58623
[job qc_to_rec] /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/8803605effa5d60a41ef8b980ea58623$ docker \
run \
-i \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa.fai:/var/lib/cwl/stg7fb523d5-e52d-4845-a42e-6f9666e39fdb/hg38.fa.fai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa:/var/lib/cwl/stg7fb523d5-e52d-4845-a42e-6f9666e39fdb/hg38.fa:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/fc82baff4ab0229ad7895229e3fdb04f/bedprep/NA12878-chr20-coverage.callable-vrsubset-callableblocks.bed:/var/lib/cwl/stg6970d96d-fc73-44b4-9c5a-77cb2dad5e8b/NA12878-chr20-coverage.callable-vrsubset-callableblocks.bed:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/5b7405849c774e668e972adec7efea63/align/NA12878-chr20/NA12878-chr20-sort.bam:/var/lib/cwl/stg44a62a2e-e16f-4b45-9c8c-5aa93787667e/NA12878-chr20-sort.bam:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.dict:/var/lib/cwl/stg7fb523d5-e52d-4845-a42e-6f9666e39fdb/hg38.dict:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/5b7405849c774e668e972adec7efea63/align/NA12878-chr20/NA12878-chr20-sort.bam.bai:/var/lib/cwl/stg44a62a2e-e16f-4b45-9c8c-5aa93787667e/NA12878-chr20-sort.bam.bai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/fc82baff4ab0229ad7895229e3fdb04f/bedprep/NA12878-chr20-coverage.callable-vrsubset-callableblocks-merged.bed:/var/lib/cwl/stg4e0a3e00-7bdb-406d-a866-c07b7d6eca40/NA12878-chr20-coverage.callable-vrsubset-callableblocks-merged.bed:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/8803605effa5d60a41ef8b980ea58623:/var/spool/cwl:rw \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/tmp/ux0sdp:/tmp:rw \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
--env=MPLCONFIGDIR=. \
quay.io/bcbio/bcbio-base \
bcbio_nextgen.py \
runfn \
qc_to_rec \
cwl \
sentinel_runtime=cores,1,ram,3072 \
sentinel_parallel=multi-combined \
'sentinel_outputs=qc_rec:description;reference__fasta__base;config__algorithm__coverage_interval;genome_build;config__algorithm__tools_off;config__algorithm__qc;analysis;config__algorithm__tools_on;config__algorithm__variant_regions;align_bam;config__algorithm__variant_regions_merged;config__algorithm__coverage;config__algorithm__coverage_merged' \
sentinel_inputs=align_bam:var,analysis:var,reference__fasta__base:var,genome_build:var,config__algorithm__coverage_interval:var,config__algorithm__tools_on:var,config__algorithm__tools_off:var,config__algorithm__qc:var,config__algorithm__variant_regions:var,config__algorithm__variant_regions_merged:var,config__algorithm__coverage:var,config__algorithm__coverage_merged:var,description:var
[job qc_to_rec] completed success
[step qc_to_rec] completed success
[job combine_sample_regions] Output of job will be cached in /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/47a652c62c00ca1a87d7c3b73b55af4a
[job combine_sample_regions] /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/47a652c62c00ca1a87d7c3b73b55af4a$ docker \
run \
-i \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa.fai:/var/lib/cwl/stg34f4d2d2-06c6-4181-b6d5-2089c8e66e77/hg38.fa.fai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/fc82baff4ab0229ad7895229e3fdb04f/align/NA12878-chr20/NA12878-chr20-coverage.callable-vrsubset-nblocks.bed:/var/lib/cwl/stg8ed16fef-83f6-4778-b004-ec90d75e286e/NA12878-chr20-coverage.callable-vrsubset-nblocks.bed:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa:/var/lib/cwl/stg34f4d2d2-06c6-4181-b6d5-2089c8e66e77/hg38.fa:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.dict:/var/lib/cwl/stg34f4d2d2-06c6-4181-b6d5-2089c8e66e77/hg38.dict:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/fc82baff4ab0229ad7895229e3fdb04f/align/NA12878-chr20/NA12878-chr20-coverage.callable-vrsubset.bed:/var/lib/cwl/stga467ddad-1b70-4c4f-be7b-197217a1e780/NA12878-chr20-coverage.callable-vrsubset.bed:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/47a652c62c00ca1a87d7c3b73b55af4a:/var/spool/cwl:rw \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/tmp/Ib5TUe:/tmp:rw \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
--env=MPLCONFIGDIR=. \
quay.io/bcbio/bcbio-align \
bcbio_nextgen.py \
runfn \
combine_sample_regions \
cwl \
sentinel_runtime=cores,1,ram,3072 \
sentinel_parallel=multi-combined \
sentinel_outputs=config__algorithm__callable_regions,config__algorithm__non_callable_regions,config__algorithm__callable_count \
sentinel_inputs=regions__callable:var,regions__nblock:var,config__algorithm__nomap_split_size:var,config__algorithm__nomap_split_targets:var,reference__fasta__base:var,description:var
[job combine_sample_regions] completed success
[step combine_sample_regions] completed success
[job pipeline_summary] Output of job will be cached in /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/e3c48addd3215bdc7954982e76974a41
['docker', 'pull', 'quay.io/bcbio/bcbio-qc']
Using default tag: latest
latest: Pulling from bcbio/bcbio-qc
75c416ea735c: Already exists
c6ff40b6d658: Already exists
a7050fc1f338: Already exists
f0ffb5cf6ba9: Already exists
be232718519c: Already exists
2834eb77be41: Already exists
469e49e2f3b4: Pulling fs layer
03df9b03023e: Pulling fs layer
469e49e2f3b4: Verifying Checksum
469e49e2f3b4: Download complete
469e49e2f3b4: Pull complete
03df9b03023e: Verifying Checksum
03df9b03023e: Download complete
03df9b03023e: Pull complete
Digest: sha256:a4f8b1f734924fabbcbed1de34e7ea9674ccf7c64a0378b763aef2ef2fc1aa5c
Status: Downloaded newer image for quay.io/bcbio/bcbio-qc:latest
[job pipeline_summary] /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/e3c48addd3215bdc7954982e76974a41$ docker \
run \
-i \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa:/var/lib/cwl/stg889364c3-2032-4731-b26a-1ab0b8c9c2d2/hg38.fa:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.dict:/var/lib/cwl/stg889364c3-2032-4731-b26a-1ab0b8c9c2d2/hg38.dict:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/fc82baff4ab0229ad7895229e3fdb04f/bedprep/NA12878-chr20-coverage.callable-vrsubset-callableblocks.bed:/var/lib/cwl/stg6e6e5530-2c15-40bb-8a41-58667987062b/NA12878-chr20-coverage.callable-vrsubset-callableblocks.bed:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/biodata/collections/hg38/seq/hg38.fa.fai:/var/lib/cwl/stg889364c3-2032-4731-b26a-1ab0b8c9c2d2/hg38.fa.fai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/5b7405849c774e668e972adec7efea63/align/NA12878-chr20/NA12878-chr20-sort.bam:/var/lib/cwl/stgc5f075e7-bfb8-422b-97fd-840c66350fd7/NA12878-chr20-sort.bam:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/fc82baff4ab0229ad7895229e3fdb04f/bedprep/NA12878-chr20-coverage.callable-vrsubset-callableblocks-merged.bed:/var/lib/cwl/stg5943fc6f-2e4e-46a5-baee-1a3e71e0c81d/NA12878-chr20-coverage.callable-vrsubset-callableblocks-merged.bed:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/5b7405849c774e668e972adec7efea63/align/NA12878-chr20/NA12878-chr20-sort.bam.bai:/var/lib/cwl/stgc5f075e7-bfb8-422b-97fd-840c66350fd7/NA12878-chr20-sort.bam.bai:ro \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/e3c48addd3215bdc7954982e76974a41:/var/spool/cwl:rw \
--volume=/home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/tmp/jEZfMH:/tmp:rw \
--workdir=/var/spool/cwl \
--read-only=true \
--user=1000 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/var/spool/cwl \
--env=MPLCONFIGDIR=. \
quay.io/bcbio/bcbio-qc \
bcbio_nextgen.py \
runfn \
pipeline_summary \
cwl \
sentinel_runtime=cores,8,ram,24576 \
sentinel_parallel=multi-parallel \
'sentinel_outputs=qcout_rec:summary__qc;summary__metrics;description;reference__fasta__base;config__algorithm__coverage_interval;genome_build;config__algorithm__tools_off;config__algorithm__qc;analysis;config__algorithm__tools_on;config__algorithm__variant_regions;align_bam;config__algorithm__variant_regions_merged;config__algorithm__coverage;config__algorithm__coverage_merged' \
sentinel_inputs=qc_rec:record
[job pipeline_summary] completed success
[job pipeline_summary_2] Using cached output in /home/ubuntu/ga4gh-challenge-test/bcbio/NA12878-platinum-chr20-workflow/cache/e3c48addd3215bdc7954982e76974a41
Workflow error, try again with --debug for more information:
Output file path /var/lib/cwl/stg889364c3-2032-4731-b26a-1ab0b8c9c2d2/hg38.fa must be within designated output directory (/var/spool/cwl) or an input file pass through.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment